New Features as of version 1.3.1 July 2011 XML conversion updates and generic speedups: * Sped up C++ XMLLoader significantly * speed_test.cc is significantly faster * string comparisons, string hashing faster * Documentation clean-up and updates speedtest.cc: New updates for PicklingTools 1.3.1 Speeds are significantly better! About 20% speedup Current speeds in seconds: (-O4 on 64-bit Fedora 14 machine) C++ Python g++ 4.5.1 2.7 # Comments ----------------------------- Pickle Text 5.90 4.82 # Python slightly faster Pickle Protocol 0 12.23 12.65 # Pickle Protocol 2 1.30 3.41 # C++ significantly faster Pickle M2k 2.98 N/A # Pickle OpenContainers 1.25 N/A # OC is fastest overall UnPickle Text 23.40 38.19 # C++ faster UnPickle Protocol OLD 0 29.53 7.13 # Why OLD P0 is deprecated! UnPickle Protocol NEW 0 7.24 7.13 # UnPickle Protocol OLD 2 8.23 3.66 # Why OLD P2 is deprecated! UnPickle Protocol NEW 2 4.34 3.66 # Python still faster UnPickle M2k 9.72 N/A # UnPickle OpenContainers 2.41 N/A # OC is fastest overall The speed of pickling2/unpickling2 in Python 2.7 = 3.41+3.66 = 7.07 secs The speed of pickling2/unpickling2 in Ptools 1.3.1 = 1.30+4.34 = 5.64 secs The round trip time of the C++ impl is about 15-20% faster than Python 2.7. C++ XMLLoader: + Sped up the XMLLoader by 8x with Eval option on (4x without): + modified OCValReader to be able to return early on failed Evals + updated Array so that resizes can be faster (and avoids deep-copies by specializing a templatized ArrayCopy) + Fixed 5 minor bugs in the XML translations + handle binary escapes + comments after/before XML header + sees (but ignores) other than xml (ie., ignores DTDs) + recongizes namespace ':' in attributes, but limited basis (full namespace support is coming in a later release) + When using XML_LOAD_EVAL_CONTENT, spaces around the front + When using XML_LOAD_EVAL_CONTENT, spaces around the front and back used to cause the EVAL to fail: not anymore + updated xmlload_test/xmldump_test to test 5 bugs mentioned + TODOS for XML: handle namespaces, DTDs better Python XMLLoader: + Fixed same 5 minor bugs as C++ XMLLoader in XML translations + When using XML_LOAD_EVAL_CONTENT, tries to use ast.literal_eval which is far safer than eval + updated xmlload_test/xmldump_test to test 5 bugs mentioned + Fixed bug in parsereader Had troubles with GCC/G++ 4.5.1 20100924 on Fedora release 14 when ANY optimization is on: put a comment in the Makefile.Linux warning people to not use FC14 with optimization (FC15 works fine with -O4 on gcc 4.6.0). RedHat 6.0 works great. + Updated opalutils.h: Used data_ instead of buffer_ (CHANGES from the C++ tunings above) + Because the Makefile.Linux64 is exactly the same as the Makefile.Linux (because the tools can distinguish between 32 and 64-bit at compile-time), make the Makefile.Linux64 be a symbolic link to the 32-bit version. We continue to update the OSF1 Makefile, but currently have no way to test it. Socket shutdown issues: Updated MidasTalker/MidasServer (C++/Python) + Under some systems, the close is not good enough to completely force down a socket. The most portable way to force a socket down is to use "shutdown" then "close". Originally, for backwards compatibility, we were only call close, but most users may want to force a shutdown when closing/cleaninup: midasthing.forceShutdownOnClose(True); // C++ midasthing.forceShutdownOnClose = True; # Python I am not entirely happy adding this in, but shutdown/close seems to be the most portable way to ensure your socket goes down in some cases. (A few consults of other system programmers and a few Googles). C++ OpenContainers 1.7.1 updates: +Added ocmove.h: "Tried" to use SFINAE techniques, but found inconsistencies among C++ compilers and platforms (G++ and Intel C++) where it works in one case, doesn't compile in another case, and compiles but gives the wrong answer. So, in the end, just specialized MoveArray for multiple datatypes. Maybe in the future we can revisit. +occomplex.h: Added MoveArray specialization for complex_8 and 16 +ocarray.h: When resizing an array, how do you move items from one array to another? With POD stuff, you can just memcpy, and same with some data structures, but most of the time you have to copy and then destroy the original. When you specialize the MoveArray template, you can do the appropriate thing for your datatype. (When rvalue-references are more available, we'll use those) +ocproxy.cc: Took inline off of some templates +ocvalreader.h: Made it so we can return "false" if an Eval fails instead of throwing an exception. In the XML tools case, this made a significant speed difference. Also made it so you don't have to copy the input, you can just refer to the original input by reference, as well as turn off the context for error messages. +ocordavlhasht.h: minor change so you can have default empty node +ocavlhasht.h: If someone from "old code" sets OC_BYTES_IN_POINTER incorrectly for the current platform, this catches it and tells you how to fix it: don't set OC_BYTES_IN_POINTER anymore! +ocval.h: Added specialization of MoveArray for Val +ocreader.h: Allow the context to be turned off so that Syntax Errors can be faster +array_test.cc: updated array test to make sure the new MoveArray specializations work +ocstringhashfunction.h: Significant speedups by using a better hash: this hash was stolen straight from Python +ocport.h: Updated version number +ocstring_impl.h: Faster op== taking advantage of int_u4 is a bigger granularity to do string compares +ocavlhasht.h: Make sure that we don't have conflicting OC_BYTES_IN_POINTER sizes. +proxy_test, avlhash_test, valreader_test, tab_test, hashtable_test outputs: outputs changed because hash function changed Misc. + pickleloader.cc: grammar errors in comments + speed_test.cc: Pickle 1 "looks like" a memory leak, but isn't: fixed + Makefile.Linux: added icc support, -g + The FAQ and User's Guide have been updated and cleaned-up: in a few places, the Python code was just wrong or confusing: those have been cleaned.