PicklingTools: Cross Language Tools for Communication
The PicklingTools library is an open-source collection of tools for allowing C++ and Python systems to share information quickly and easily. From both C++ and Python, it is easy to create socket servers and clients (TCP/IP or UDP) as well as read/write data files in many different serialization formats. There is support for X-Midas, Midas 2k, XMPY, raw C++ and raw Python.
At the end of the day, the PicklingTools is an answer to the question: "How can I pickle in C++?" And Java.
Version 1.7.0 is available. Updates precision, and fixes problem with const Val array access.
Version 1.6.3 is available. Made sure works with CentOS 7, added C++ keys(), values(), items() so more Pythonesque.
Version 1.6.2 is available. Bug fix release, update to Array for allowing it to reference other memory without owning it.
Version 1.6.1 is available. This is a very small release, just fixing a problem with FixedSizeAllocator so it wouldn't hang when using cross process shared memory.
Version 1.6.0 is available: This release adds OC Serialization to Python via a Python C Extension module: this allows faster serialization and the ability to serialize numpy and strings over 4Gigabytes. This release also adds significant fixes to shared memory tools, as well as complex integers for Vals.
Two talks are available now from C++ Now! (2013) related to the PicklingTools work:
Dynamic, Recursive, Heterogeneous Types in Statically-Typed Languages and
Thread-safe and Thread-neutral Bags.
From PyCon 2011, a copy of the of the slides from talk "Everything You Wanted to Know About Pickling, But Were Afraid To Ask!" is available here in PDF. The talk is also available on blip tv.
A paper giving the history and a high-level overview of the PicklingTools was presented at New Application Areas in Open Source Software (NAAOSS).
If there had been a PyCon 2010 Tutorial/Talk, these would have been the slides.There was, however, a poster.
A Technical Report describing some of the decisions of the OpenContainers is available here: this is a technical paper describing how a threaded environment influences decisions of a containers library.
The PicklingTools is released under a BSD-style license: Essentially, free free to use to it as you wish, just please give us credit. See License for more details.
To allow C++ and Python to talk, they need a common "currency": that currency is the Python dictionary. For example:
{ 'key1':323.1, 'key2': 'ping' }
When a Python dictionary is serialized (and written to a socket or saved to a disk), this is known as pickling in Python-speak. Thus the name PicklingTools.
The major goal of the PicklingTools is make it easy to write socket servers and clients (TCP/IP or UDP) in both C++ and Python.
One minor goal is to give C++ users a "Python-like" experience with minimal fuss. See the FAQ for more details.
The original goal of the PicklingTools was to bridge legacy applications (written in Midas 2k or X-Midas) with Python. All different types of systems can communicate easily: