Python’s xml.dom.minidom.toprettyxml has a feature/flaw that renders it useless for many common applications.
Someone was kind enough to post a hack which works around the problem. That hack had a small bug, which I’ve fixed; you’ll find the revised code below.
METAUPDATE Since writing this post, I’ve started using lxml for all my xml processing Python. Incidentally, it fixes the pretty printing problem with toprettyxml, but the actual reason I switched was for performance: lxml parses xml significantly faster (an order of magnitude faster) than minidom. So my new recommendation is: consider using lxml instead of minidom.
toprettyxml adds extra white space when printing the contents of text nodes. This may not sound like a serious drawback, but it is. Consider a simple xml snippet:
This Python script:
# python 2.4 import xml.dom.minidom as dom myText = '''<Author>Ron Rothman</Author>''' print xml.dom.minidom.parseString(myText).toprettyxml()
generates this output:
<?xml version="1.0" ?> <Author> Ron Rothman </Author>
Note the extra line breaks: the text “Ron Rothman” is printed on its own line, and indented. That may not matter much to a human reading the output, but it sure as hell matters to an XML parser. (Recall: whitespace is significant in XML)
To put it another way: the DOM object that represents the output (with line breaks) is NOT identical to the DOM object that represented the input.
Semantically, the author in the original XML is
"Ron Rothman", but the author in the “pretty” XML is [approximately]
" Ron Rothman ".
This is devastating news to anyone who hopes to re-parse the “pretty” XML in some other context. It means that you can’t use minidom.toprettyxml() to produce XML that will be parsed downstream.
UPDATED If you’re in a rush, skip ahead to the best solution, #4.
- calling toprettyxml with “creative” (non-default) parameters
1. Don’t use minidom
There are plenty of other XML packages to choose from.
But: minidom is appealing because it’s lightweight, and is included with the Python distribution. Seems a shame to toss it for just one flaw.
UPDATEI’ve started using lxml and I highly recommend it as a replacement for minidom or PyXML.
2. Use minidom, but don’t use toprettyxml()
Use minidom.toxml(), which doesn’t suffer from the same problem (because it doesn’t insert any whitespace).
But: Your machine-readable XML will make heads spin, should someone be foolish enough to try to read it.
3. Hack toprettyxml to do The Right Thing
Replace toprettyxml by using the code below.
But: It smells. Like a hack. Fragile; likely to break with future releases of minidom.
On the other hand: It’s not that bad. And hey, it does the trick. (But YMMV.)
def fixed_writexml(self, writer, indent="", addindent="", newl=""): # indent = current indentation # addindent = indentation to add to higher levels # newl = newline string writer.write(indent+"<" + self.tagName) attrs = self._get_attributes() a_names = attrs.keys() a_names.sort() for a_name in a_names: writer.write(" %s=\"" % a_name) xml.dom.minidom._write_data(writer, attrs[a_name].value) writer.write("\"") if self.childNodes: if len(self.childNodes) == 1 \ and self.childNodes.nodeType == xml.dom.minidom.Node.TEXT_NODE: writer.write(">") self.childNodes.writexml(writer, "", "", "") writer.write("</%s>%s" % (self.tagName, newl)) return writer.write(">%s"%(newl)) for node in self.childNodes: node.writexml(writer,indent+addindent,addindent,newl) writer.write("%s</%s>%s" % (indent,self.tagName,newl)) else: writer.write("/>%s"%(newl)) # replace minidom's function with ours xml.dom.minidom.Element.writexml = fixed_writexml
I just copied the original toprettyxml code from
/usr/lib/python2.4/xml/dom/minidom.py and made the modifications that are highlighted in yellow. It ain’t pretty, but it seems to work. (Suggestions for improvements (I’m a Python n00b) are welcome.)
[Credit to Oluseyi at gamedev.net for the original hack; I just fixed it so that it worked with character entities.]
UPDATE! 4. Use xml.dom.ext.PrettyPrint
Who knew? All along, an alternative to toprettyxml was available to me. Works like a charm. Robust. 100% Kosher Python. Definitely the method I’ll be using.
But: Need to have PyXML installed. In my case, it was already installed, so this is my method of choice. (It’s worth pointing out that if you already have PyXML installed, you might want to consider using it exclusively, in lieu of minidom.)
We just write a simple wrapper, and we’re done:
from xml.dom.ext import PrettyPrint from StringIO import StringIO def toprettyxml_fixed (node, encoding='utf-8'): tmpStream = StringIO() PrettyPrint(node, stream=tmpStream, encoding=encoding) return tmpStream.getvalue()
One lesson from all this: TMTOWTDI applies to more than just Perl.
Please–let me know what you think.