New Features as of version 1.4.1 November 2012 Top-Level: * Added Python C-extension module for converting between dicts and XML. There is now a "cxmltools" module which uses the C++ code to do the conversions between dicts and XML: dict to XML is 10x faster, XML to dict is 100x faster. * Changed default top_level_key of all dict -> XML conversions from None to "top". *This is a minor interface change*! Why? Using "top" caused too many user problems (the output XML had no outer level key, and would cause confusion with other XML tools). It was a bad default: it would confuse users. This affects a bunch of routines: C++: WriteValToXMLString, WriteValToXMLString, WriteValToXMLStream, WriteValToXMLFile, WriteValToXMLFILEPointer Python: WriteToXMLStream, WriteToXMLFile, WriteToXMLStream Python C Ext: CWriteToXMLStream, CWriteToXMLFile, CWriteToXMLStream This change shouldn't affect users too much: Almost *all* of the example programs show canonical use of using "top" anyway as the top-level key: this just makes it the default. This really only affects the convenience routines, all of the XMLDumper examples in the documentation explicitly set all of the options anyway. * int_n and int_un interface changes: we *should* change major version number for PicklingTools, but these interface changes fix broken behavior anyway. Problem: When working with int_un/int_n classes with normal classes, conversion were happening the wrong way: we would downconvert big ints to small ints instead of the other way around. Three major changes: (a) added construction of int_n/int_un from string: i.e,, int_n i = "100000000000000000000000000000000"; (b) Allowed mixing of ints and int_n/int_un correctly:, i.e. int_n i = int_n("100000000000000000") + 100; Now works and works as expected. Previously, it would compile but DOWNCAST the int_n to an int_8 instead of UNPCONVERTING the 100! Currently, the 100 gets upconverted to an int_n, just like expected. (c) Disallowed casting out from int_n to int_8 (sim.for int_un to int_u8). Too many problems using that with mixed mode int/int_n, as the downcast would confuse the overloading, and many things that SHOULD compiled would complain there were multiple overloads. I.e., int_un in = "1000000000000000000000000000000000"; // ok int_u8 i = in; // WILL NO LONGER COMPILE! To get around this. use "as()": int_un in = "1000000000000000000000000000000000"; // ok int_u8 i = in.as(); // ok: new way to get value This is probably the biggest interface change. (If we had C++11 everywhere and the "explicit" for user-defined conversions, we wouldn't need to do this. As it is, this is the best way to handle this interface change untill C++11 is ubiquitous). (d) Minor updates to int_un/int_n work with the serverside, middleside and clientside shared memory servers across processes. Details: * C++: Updated to OpenContainers 1.7.5 * Added FILEPointerReader so we can deal with FILE* as input for XML * Updated ocport.h to add a new switch so that "long" can be supported with Val, i.e.: Val v = long(1); long vl = v; This works on some platforms, but not all. We discovered when mixing with Python (where longs are everywhere) * Enabled using long, unsigned long, long long, unsigned long long with Val. Any conversions to and from these types will now work correctly from Val. I.e., "Val v = long(1); long ll = v;" should work fine (before you would have to change to one of int_4 or whatever). * Updated the int_n and int_un to work with strings better: before the syntax was "int_un a = MakeBugUIntFromString("123");" Now, we can use "int_un a = "123";". This is a delicate balance of overloaded constructors and Val overloading. * Updated a bunch of tests to test these features and make sure we didn't break anything. * Major surgery on BigUInt and BigInt to fix conversion problems and mixing ints/int_un (see major headlines above). Fixing these required adding a bunch of default constructors for all different types, instantiating all the mathops and comp. operations (so they could participate in overloading better), and removing the user-defined operations. * Also cleaned-up BigUInt and BigInt to work better with the shared memory allocator pool. * Updated array so that 0 length arrays DO NOT allocate resources: this helps speed up the empty array construction and destruction * XMLLoader: * Added ReadFromXMLString and ReadFromXMLFILE, partly for completeness and partly so the Python C-Extension module can deal with Python style streams (which are FILE*) * XMLLoader: * Added WriteToXMLString for completeness for writing to strings * Added PythonCExt directory: This directory contains the Python C extension module. To build the C Extension module, it includes much of the code from opencontainers1_7_5 and the C++ area. * The approach taken by this module is an "adapter" approach: rather than rewrite all the C++ routines in Pythonesque C, we reuse the code and "adapt" it by converting from PyObject* to Val (and vice-verse) to use the C++ XMLDumper and C++ XMLLoader. The conversion between the two object modules (Val/PyObject) is surprisingly cheap: it's cheaper to convert from PyObject* to Val and convert from Val back to PyObject* than deepcopy! With this approach, we get to reuse all the C++ goodness for the XMLTools. * Added xmltimingtests.py so we can see how expensive the C Extension module and the Python modules are. As of this date, the Python C Extension module for the XMLdumper is about 6x faster and the Python C Extension for the XMLLoader is about 60x faster. * Added support for NumPy and Numeric inside the C Extension modules. Because when this module is compiled, it's difficult to tell if Numeric or NumPy is supported, we use dynamic techniques (import and see if it fails) to check for Numeric and Numpy support. * Added plenty of testing (convert_test.py) to make sure we can convert between Val and PyObject and preserve information: this includes preserving structure like deep_copy would. * Added a README to demonstrate usage and how to build * Added documentation to the Docs area for the C-Extension module * Python * Added cxmltools.py: this is the "wrapper" around the pyobjconvert Python C-extension module that makes the conversion tools look like the xmltools.py module. * Moved the definitions of the options for both the XMLDumper and XMLLoader to xmldumper_def.py and xmlloader_def.py so that the xmltools.py and the cxmltools.py could share the definitions * Updated the xml2dict.py and dict2xml.py to try to use the cxmltools (if they are available), otherwise use the pure Python version * X-Midas * Added the pythoncext directory to the X-Midas tree (with everything underneath it as in PythonCExt). * Updated the builtopt.txt macro to build the Python C extension module. Because X-Midas inserts both "python" and "python/lib" on the X-MIdas PYTHONPATH automatically, the C extension module "pyobjconvert.so" is inserted into "python/lib" directly. * Updated Python XMLLoader so it worked better with complex->string compares (in Python 2.4)