Comments on: Showdown - Java HTML Parsing Comparison

By: Tom

Tom — Mon, 16 Jun 2008 22:22:48 +0000

Hi,

thanx for the article. Unfortunately, I am not able to use NekoHTML with Saxon. Saxon always crushes down with an error that the document ins’t valid. Do you think you could leave there an example of your code?

Thank you

Tom

By: Ben

Ben — Sun, 13 Apr 2008 19:35:21 +0000

Bambaria,
I don’t see your results anywhere. Did you try any of the links I published? I found that most of the parsers did not handle these URLs well. The Mozilla Parser is the best I’ve come across for malformed xml.

-Ben

By: Bambarbia Kirkudu

Bambarbia Kirkudu — Fri, 11 Apr 2008 14:48:48 +0000

“Only URL to nice internet shop (for beauties!) shows the difference, 144 links found with HtmlCleaner, and 116 with NekoHTML. After quick copy-paste to Excel and sorting links I found that some links are simply repeated by HtmlCleaner probably due to bug… so that all parsers behave the same, correctly parsing ugliest HTML.”

By: HTML Parsing Showdown - New Contender Takes Title | Lumidant

HTML Parsing Showdown - New Contender Takes Title | Lumidant — Fri, 21 Mar 2008 20:51:26 +0000

[...] of my first posts was a comparison of HTML parsers. Today I found a particularly challenging document to parse. None of the parsers I had compared [...]

By: casino juegos de azar

casino juegos de azar — Sun, 10 Feb 2008 09:44:35 +0000

Nice! We rather appreciated the website

By: Ben

Ben — Fri, 08 Feb 2008 00:13:11 +0000

Hi Lance,
You are correct about the TagSoup namespace issue and I’ve updated the code above to reflect an easier fix.
Since I didn’t post the actual TagSoup results I was going to rerun the test and share the results with you. Unfortunately, I’m afraid I’ve changed the code in some manner since I’ve written this post. I’m not able to get very far since I’m now being presented with an exception: “org.w3c.dom.DOMException: NOT_FOUND_ERR: An attempt is made to reference a node in a context where it does not exist.” I’m not going to spend any time debugging this since HTML Cleaner seems to work well enough for me, but if you’ve run into the same problem and know what I’m doing wrong I’d be happy to take another look.

-Ben

By: Cemo

Cemo — Thu, 07 Feb 2008 19:56:26 +0000

Very nice article Ben. I was looking for such a helpful post.

By: Lance

Lance — Wed, 06 Feb 2008 20:53:20 +0000

I do like how HtmlCleaner just required a constructor taking a URL, rather than all the extra code the others require. I will look into it.

By: Lance

Lance — Wed, 06 Feb 2008 20:50:58 +0000

I’m curious about your TagSoup test, as I’ve had good results with it. One thing I found was that it actually created a namespace for the nodes, so I had to preface my xquery with “h:”. Following is from some code I used to print out the movies in my Blockbuster Queue (of course I wrote helper code - not included):

Dom4jXPath xpath = new Dom4jXPath(”//h:div[@class='title']“);
xpath.addNamespace(”h”, “http://www.w3.org/1999/xhtml”);
java.util.List divs =xpath.selectNodes(”//h:div[@class='title']“, doc);
…
later
…
Dom4jXPath xpath = new Dom4jXPath(”h:a”);
xpath.addNamespace(”h”, “http://www.w3.org/1999/xhtml”);
xpath.stringValue(element);

By: Ben

Ben — Wed, 06 Feb 2008 03:12:49 +0000

Hi Anjan,
This method will easily beat the manual one. I use Saxon to run the actual XQuery on the DOM once I’ve received it back from the parse operation. Perhaps if there is interest I can write a post on how to use Saxon.

-Ben