Common Vision Blox: Prototyping with Python
Prototyping is an important part of developing machine vision solutions. In addition to allowing developers to explore and learn about the problem in hand, it enables interim solutions to be presented to a customer or stakeholder to make sure everything is going in the right direction. In this way, everyone can see how the proposed solution would work and understand how a project is progressing with the added benefit that a better estimation of time and cost for the final solution can be provided.
www.youtube.com/watch
The video demonstrates interfacing with the UI and shows how easy it is to set up a single stream image display from a device with just 26 lines of code and link it to a UI.
Speeding up the prototyping process is extremely beneficial to everyone involved. Three new object-oriented APIs, supporting C++, .NET and Python have been introduced in CVB2019 to provide a new way of developing machine vision solutions with Common Vision Blox. The new APIs are compatible with the existing CVB API, meaning that users can build on existing applications without having to re-write their codebase.
The benefits of using Python
CVBpy brings the power and speed of CVB to a new group of users who are used to quick results and easy programming using Python. Python is a free, open source, interpreted, high-level, general-purpose programming language. Easy to learn and simple to use, Python is one of the most widely used programming languages.
A recent survey showed that Python is not only one of the most loved languages by developers and but is also the most wanted language. It is portable so there are no concerns over compilers and hardware and it can run on embedded, Linux and Windows platforms. Because it is high level and so easy to learn it is possible to do many things very quickly, making it highly productive and therefore particularly suitable for prototyping. There are already Python frameworks in use in many application areas, including IoT, Machine Learning, Deep Learning, and Artificial Intelligence meaning that interfaces are available for an enormous range of libraries.
Combining CVB and Python
To make the most of CVB’s impressive capabilities, it was essential to create an optimised CVB Python binding. The general way of producing a Python interface is to generate it from a C interface using readily available tools. However because Python can do so much more than C this is not the best approach since it omits a lot of powerful Python features. In principle, it is better to generate the Python binder from a more modern object orientated, potentially powerful architecture like C++. Although this is an improvement, it still lacks some of Python’s special features since the generator takes one language and squeezes it into another, ignoring some of the key characteristics of both languages.
The outcome usually contains the smallest set of common features, which is far from what a developer needs. The solution is to manually create the wrapper. Although this seems complex at first sight the development team at STEMMER IMAGING, with an intimate knowledge of both CVB and Python, were able to do so quickly and efficiently.
The resulting CVBpy binder provides access to a lot of special Python features that can be used for programming and prototyping with CVB. These include:
- Explicit resource management. Python takes care of the management of resources such as memory or files and CVBpy supports this. This avoids the need to manually free up memory or close files or outsource the management of these resources. Explicit resource management is particularly important for imaging applications involving the acquisition or processing of large images and the use of high frame rates and or a large number of different devices which generate large amounts of data and require huge amounts of memory.
- Built-in documentation. Python features built-in documentation, so with CVBpy an IDE will display documentation help for each and every function. This is built into the library itself so there is no need to install separate documentation.
- Improved multithreading and async support for manual activities. This allows native threads to be started and the use of the asynchronous feature of Python, which is essentially working through an event handler.
- Stable API. Python offers a stable API, or ABI (Application Binary Interface) over the lifetime of Python 3, thanks to PEP 384. This means that CVBPy will run on every version of Python, from V3.5 to the current V3.8 and beyond with no adjustments required for future Python 3 versions.
- CVBpy provides an interface to Python’s NumPy. NumPy adds support for large, multi-dimensional arrays and matrices, together with a large collection of high-level mathematical functions to operate on these arrays. The CVBpy interface allows images that have been acquired from a real device to be transferred directly into NumPy where any algorithms that have been developed there can be applied.
- UI using PySide2. Imaging applications require a suitable display capability to show the original images and the effects of any algorithms that have been developed. As Python does not come with any user interface itself, CVBpy includes an interface to PySide2 which is a Qt5 wrapper for Python for creating UIs.
The structure of new object-oriented CVB APIs, supporting C++, .NET and Python
Object orientation
CVBpy can interact with a number of basic classes from an object orientation point of view. The main entry point for accessing hardware devices, such as the camera, is the device factory, which also provides access to different types of devices (see Figure above). There is a general interface to video devices with real hardware behind them or to non-streaming devices that just have a control path. There are also emulated devices with defined data – either as a classic media file or in an emulation file format.
Devices can have one or multiple streams and the use of multiple streams is under development but there is usually at least one stream available. On this stream there is a ring buffer into which images are acquired. This stream provides the most recent image for processing. The image itself consists of multiple planes. The device itself has node maps such as a GenAPI node map for the control path. At different levels, the maps and the nodes and the features can be obtained from each device which describes itself through an XML. In addition there are some convenient interfaces on the devices for easier access of common features.
This simple CVBpy ‘hello world’ example shows the code required to acquire a number of images.
Practical examples of CVBpy
One of the simplest examples is the Python ‘hello world’ program. This CVBpy ‘hello world’ example shows the code required to acquire a number of images (Figure above). It is quite an easy piece of code. The first action is to open a mock up device from the device factory and get a stream from it. The stream is then started and acquires 10 images. The status of each image is checked to make sure that it is OK and then the number of the image is printed out. To clean up afterwards, the acquisition is simply aborted.
For more advanced examples, a live display would be required that is more responsive and adaptable. The UI application needs to be multithreaded to allow the camera to run asynchronously to the UI. Generic handling of acquisition hardware is also required and for best practice the UI should be separated from the logic code.
www.stemmer-imaging.com
