Geeklog Forums

The above blank posting actually shows the posting below to me but when submit pressed it comes up as blank. I cut and pasted it 2nd time it showed up.


Thanks ... basically I am using the Professional theme and as a matter of interest just set up an out of the box setup of 1.3.11 and run that through HTML Validate and get similar errors so it would seem that the theme has these errors already in place .... which is not good. Did you run it on Zim as my validator is throwing up also many errors. If you are running a site based on the Professional theme perhaps a similar test on that may also throw up errors.

The page size is not much more than that this site and google search do not run away from its size and within a hour google search (not news) have the article referenced. Although I consider the error issue very serious and I will see it is addressed early next week.

I sincerely thank you for running validate, something I should have done and not took it for granted that the Professional theme was error free.

Regards

Tess

I have just run the W3 validator and would make these comments ... 90% of the errors you relate come from the advertising bannercode which uses FRAMES. The majority of the other errors are actually caused by character problems, the origin of our text is Asia (i.e.) non SGML character number 147.

Our articles are emailed in from journalists using macs, pcs and such with the language set to who knows what. Our server on the other hand is set up correctly for US English/English in consequence differences in character codes throw up on a regular basis ... so I have discounted this type of error. The other 5 or 6 are sloppy code either by the theme or ourselves and we shall address them.

Funny news.google have never said the problem is W3 errors they concentrate only on URL problems I have written to the head of news.google who I have been dealing attaching a copy of your posting and will come back when he comments.

Thank you Dirk

I will get someone to check the addition of extensions in lib.common and see what happens. Not sure if a change at this time with 25000 stories in the archive would make sense as most would be without extension others with.

We have checked the errors most as said are language the 5 or six with real errors some caused by us and some in the theme templates have been corrected.

news.google say it is not our page size average 63 which is the problem as compared with CNN and BBC ours is quite compact. They also say the W3 errors should not raise a problem they are still ademant the problem is how GL produces the URL's. AS GL seems to be the only php type CMS that is not conforming to convention it is not worth news.google adjusting the crawler to conform, as they are aware we are the only major organization, (their terminology) using GL with such an input and output.

Our page hits for October were just short of 1.8 million with 700 plus posted news articles. 95% of our articles are emailed into GL through Sendmail and straight into the mysql and as stated originate from a variety of users set at who knows what language.

It is because of our unique content that news.google have spent so much time trying to get everything to work.

Dirk if you are interested I can send you the various technical responses google have sent ... but as I say the alterations needed are just not work the effort from GL development point of view.

Despite the problems Dirk I am sticking with GL - my additions to this thread were not by way of compaint but just to respond to our Zimbabwe collegue who I felt was wasting his time to expect news.google to index his site after the problems we have had.

Thanks as always Dirk.

Marites

Thanks DIRK for your answer. I think Google News want a .html at the end of the Story ID, thats what they are saying exactly. Now have been to COM_makesid is lib.common and this is what i got.




Now i need to add the .html automatically in COM_makesid so that i wont have to add everytime i post a story manually. PLEASE HELP ME WITH THE HACK.


Thank you in advance

I have to correct myself here: Changing COM_makesid would, as mentioned above, also change the IDs used for links and events. The problem there is that those are too short (20 characters, while an ID with attached ".html" would be up to 23 characters long).

So, plan B is to change it where COM_makesid is called. In admin/story.php: And in submit.php: and a second time:
Note that there are further calls to COM_makeSid in submit.php, but those are for events ($A['eid'] = ...)and links ($A['lid'] = ...) and should be left unchanged.

bye, Dirk

If you have any definitive information about what it is that Google News doesn't like about Geeklog's URLs, I'd like to hear it. At the very least, we could make an FAQ entry for it.

bye, Dirk

Thanks DIRK, am trying to understand where u say and a second time.

Where do i add that?

Thank you in advance

Will put the relevant emails together and mail to you personally early next week.

Regards

- Tess -

Munazip ... Earlier this year on a test site we hand added .html extensions to about 50 articles - google still could not read them - I see you have added an extension to your first story on your site it will be interesting to see if news.google say they can read it or not.

- Tess -

Just search for "makesid" (or "makeSid") in the two files and you will see.

bye, Dirk

Thankd Tess and DIRK, have done the necessary amendments but for some reason am failing to see where i can add this

It looks like it works fine without the line above.


And the aother thing is my images now have a .html extension as well, will that affect anything??

This should be the second hit for "makeSid" in submit.php (assuming you're on Geeklog 1.3.11sr2 - haven't checked with any older versions). It's called when normal users (not admins) submit stories.



I have to admit that I didn't think of this, but then again, they should have two extensions now (20051105215730123.html_1.gif etc.), and that should be fine.

bye, Dirk

The google bot can crawl through bad html, but its always best to have good html. You never know what will make the crawler fail. My site LinuxElectrons has no problems with news.google.

The only major difference between my site and your site, is your story page still has the story title as a url. It's very hard to believe that the crawler wants a .html extension at the end. If this is true, it must be due to your story page title being a URL, instead of a real story title:

Yours: My Story

Mine: My Story

Byte

I just did a search I find the uptake news.google of your stories is no larger than ours - I counted one from your front page. As I stated earlier news.google can pick up some items but not all. Putting an extension will make no difference what so ever we set up a test site earlier this year and each url had an html extension and the pickup rate was 2 or 3 from the 50.

May be we are all blaming the URL when maybe it is something else - I have tried one part story to a page and 20 and that makes no difference. I spent weeks with Google trying to find out what was the problem and at no time could 100% of the articles be picked up in a crawl. At Googles suggestion I set up the same test site in Mambo (the paid version) and there was no problem every item was picked up as it was with ArticleLive and others.

Sadly I am not a programmer so I can only stab in the dark at solutions and causes and no doubt all are wrong.

Could there be something between the URL on (we'll call it) page 1 and the link to the full article as I do find the smaller articles which don't require a 'Read More' do get picked up every time.

I reiterate google.com has no such problem and every article is picked up - it sounds crazy but what is the difference between the crawler on google.com and news.google.com.

- Tess -

Aha, it is alway the big guy that calls the shot!

See, as you said, CNN and BBC have no problem picking up the news. But Google never thought about spanking their bot for not picking up the news faithfully. Isn't it a little arrogant that they would rather try to fix your Geeklog, than look into their own bot?

Well, I feel for your problem, so I am just joining the rant. Sometimes Google deserve a stick.

Sam

Google News only retains news for 30 or 60 days. I can't remember exactly which one. I just did a news.google.com search and got:

Results 1 - 10 of about 333 for linuxelectrons

Search URL.


On Yahoo I get:
NEWS STORIES - Results 1 - 10 of about 62 for linuxelectrons.

I did a search on your domain, and not one Search engine has you listed, except for the one link on www.google.com.

I even did a search on ww.google.com.ph, and there is no links. Its probably has something to do with all that java script breaking the crawler, got to be.

Byte

Try this URL

Look here



I guess I dends on the search criteria.

It's 30 days btw.

Tess

Whether it is because I am of the opposite sex but everytime I get involved in a thread on this forum I receive disgusting suggestions from an/other member saying that should not be running a web site but a home instead. (Kept polite)

I know who this is as does he ... he has participated in this thread.

For everyones interest including his, Balita is a part-government owned site that has been online for 11 years now and yes we are able to work with people at news.google.

We are not a site that needs to make itself known to every search engine nor are we after accolades. We get that from our visitors.

I participated in this discussion to help a fellow GL'er who was experiencing similar problems to what our site had experienced.

Zimdaily does not carry Java which was suggested as the reason our site was not crawled .... as I pointed out our site was crawled hourly but it was the URL's that could not be captured in the same problem as Zimdaily had.

news.google have provided me with the explanation why our site cannot be crawled and it is not html errors, Java or page sizes I understand the same reason applies to ZimDaily (if he contact me proivately I will explain).

Enjoy your little chauvanist clique - I for one will read but will not participate in this forum again.

A million thanks go to Dirk and all the other polite members of the forum I have enjoyed their company.

Marites

You should inform Dirk so he can take care of this problem.


Hmmm....looked like you were asking for help.

For others Google says this:



Don't let one bad apple spoil it, your posts brought out discussion which has never before been discussed, which is great!

I've been looking at this lately because these posts got me thinking again. For awhile I was diving into SEO technique's. I spent quite a bit of time on research. In the end, trying to be crafty required lots of time monitoring various search engines with very little payback. In one of Google's FAQ's they say don't try to get too crafty, just build a good website and they will come.

At any rate, a previous post in this thread mentioned that Google requires a ".extension" at the end of a URL. I still find this hard to believe, why would such a high tech outfit as google, have their bread and butter algorithm fail because of a .extension. I guess its possible. I'm just in shock and disbelief. However, lightly scanning through some URL's at news.google, most of them have a .extension.

I also found another reference about the landing page should not be a URL but a h_tag: