Mallard for CRIS at ISOLDE
Mallard (Multi-Analog LossLess Acquisition of Resonance Data) is data acquisition software for the Collinear Resonant Ionization Spectroscopy experiment at CERN.
This document also exists in pdf format
Contents
Resources
Github Repository: https://github.com/kieranrcampbell/mallard
Documentation: http://kieranrcampbell.github.io/mallard/
Sourceforge page: https://sourceforge.net/projects/mallardforcris/
Introduction
Etymology
Two things frequently occur at CERN: everything is an acronym (CERN, LHC, ATLAS, ALICE, ISOLDE, CRIS, ...), and the DAQ/duck joke is made (a yellow rubber duck sits on the data acquisition desk in the ATLAS control room). The obvious challenge then for any self respecting summer student is to combine both of these, and so came Mallard: Multi Analog LossLess Acquistion of Resonance Data
Purpose
Mallard is designed for efficient data acquisition and experimental control on the CRIS experiment at ISOLDE at CERN. The software interfaces a National Instruments USB-6211 card that connects to the lasers, counters and trigger. The experiment requires the software to scan across a range of voltages set on an analogue output, and at each voltage measure counts (from the MCP) and an analogue input (relative intensity of the laser). The user selects the voltage range across which to scan, how many volts should be set in a given interval, and how many scans across the full voltage range should be performed. The timing of the voltage change comes from an external trigger that is also connected to the scanning laser.
During the data acquisition the software continuously graphs the current count rate and analogue input voltage measured, as well as the average so far in order to provide the user with an idea of whether the measurements are drifting from normal. The raw data of counts and analogue input measurements is collected and saved to disk, as well as an integrated file showing the measurements at each voltage averaged over a number of scans.
Installation
Mallard is written in python and so requires a python distribution such as Anaconda.
Dependencies
Mallard has the following dependencies:
Installing SciPy on Windows can be tedious, so use Anaconda which comes with it pre-installed. Don't install wxPython using easy_install
or pip
as these install some bizarre empty package. Instead, download the windows installer directly from their website (see above).
NIDAQmx
NIDAQmx from National Instruments provides the base set of drivers to interface the card. The full set of functionality is only available on Windows. At time of writing the latest version was 9.7 which can be downloaded from http://joule.ni.com/nidu/cds/view/p/id/3811/lang/en. NIDAQmx installs a header file containing all the C library function definitions. The location of this file must be noted and an installation file in PyDAQmx modified accordingly (see PyDAQmx documentation for details).
Mallard
From source
The source code can be downloaded (as zip or tar.gz) from the github repository. Extract it to a convenient folder then, from the top level mallard
directory it can be installed by python setup.py install
. Then mallard can be launched in the usual way, or if you prefer to use it within a python script itself it can be called using import mallard
etc.
Windows installer
A windows installer for the python package can be downloaded from the sourceforge page which will install it as a python package.
System requirements
Due to the availability of NI-DAQmx Mallard only runs on windows. Despite all data being held in RAM, during system tests it was found that 500 scans with 50 intervals per scan only takes up about 150MB, so this is certainly not a limiting factor.
Use
Basic Capture
To begin Mallard, navigate to the root mallard folder, and start by
python -m mallard.main
Due to a multiprocessing bug under python on windows we can't have the usual main.py folder to begin the module as python mallard
.
Each acquisition session of data corresponding to a particular set of settings is known as a capture. Each capture is represented by a tab in the GUI, and a new capture can be opened by File -> New Capture. Captures can be saved (see below), but cannot be reloaded into Mallard as there is currently no data analysis functionality. However, the settings for a new capture can be loaded from an old capture by File -> Load Capture then selecting the capture you wish to open.
The capture begins by pressing 'Start Capture', and can be stopped by selecting Capture -> Kill Capture. The current voltage and MCP count is displayed on the screen, as well as the scan average superimposed on the graph. The graph can be saved at any time using the Matplotlib functionality on screen.
The graph style can be changed at any time by going to File -> Preferences. The graph style can be set separately for the count graph and for the voltage graph. There are three different types:
- Step (histogram): Essentially bins the data exactly like a histogram
- Line: joins consecutive data points using lines
- Points: Each data point is represented by a separate point. For the count graph this has error bars, which are given by root(N) for N counts.
In these preferences you can also set the style of the 'mean'. Selecting 'normalised' graphs each count divided by the number of scans, whereas 'cumulative' gives the sum of all counts across all scans.
Plots can be saved using the controls on the plot itself. To move the plot around, select the 'Pan' symbol and drag with the left mouse click. To zoom in and out, both horizontally and vertically, select the 'Pan' symbol and drag with the right mouse button.
Key Concepts
- Voltage minimum: The voltage from which the laser starts scanning
- Voltage maximum: The voltage to which the laser will scan
- Intervals per scan: The number of intervals to scan in the voltage range
- Scans: The number of full scans to do (from minimum to maximum across the range)
- Clock cycles per interval: The number of onboard clock cycles to wait per scan (known as the 'rate' in NI terminology, 100 is a good number)
- Analog input: Analog input channel (default /Dev1/ai2)
- Counter input: Input channel for counting events. This is a 'virtual' channel, which requires looking at the NI handbook, but Dev1/ctr1 corresponds to Dev1/PFI1 on the 6211
- Analog output: Analog output channel (default /Dev1/ao0)
- Clock input: Input channel for the pseudo-trigger clock
File Output
When mallard runs it creates two datasets: every count measurement at every interval across every scan, and likewise for the analog input readings. These are then averaged over the scans to give a third 'integrated' dataset. As such, mallard outputs 3 data files. If the capture name is 'mycapture' then these files are:
- mycapture.integrated.csv: contains the integrated (final) dataset across all scans
- mycapture.raw.counts.csv: contains the raw count data for all intervals and scans
- mycapture.raw.ai.csv: contains the raw analog input information for all intervals and scans
Each of these files is in comma separated value format and contains a header, denoted by '#' at the beginning of each line, containing the settings listed in key concepts above. As such, any file can be used to recover capture settings for a new capture.
Mallard doesn't yet have any in built analysis, but with the data in this format it should be easy enough. The data can be opened directly in excel, or using numpy in python as follows:
import numpy as np
data = np.loadtxt('mycapture.integrated.csv', delimiter=',')
voltage_intervals = data.T[0] # need to transpose matrix
counts = data.T[1]
ai_readings = data.T[2]
and voltage
, counts
and ai_readings
can be analysed using any of the standard tools in SciPy.
Development
Mallard is written in python and is under git source control.
Package Layout
The mallard package is organised into several sub-packages:
mallard.core
This provides base functionality for the package including data and file handling. CaptureSession
represents an abstract (ie has no attached daq interface or gui), and manages calls for file management and data acquisition. SessionSettings
represents the settings for a given capture session, including input, output and counter channels. CaptureSession
holds the 'master' copy, though since everything in python is a pointer all copies (held by all mallard.core
modules). However, after settings are changed some things must be recalculated (such as voltage intervals), so if a module uses SessionSettings
it's not okay to assume changes elsewhere will be safely reflected in the current location. DataManager
holds runtime data and has the queue.get()
method to receive data from the interface. It also calculates the averages across scans, and calls the GraphManager
(which provides the interface between mallard.core
and the gui package). FileManager
deals with file input and output. It can generate the header from SessionSettings
and can read in and parse settings and filenames.
mallard.daq
This contains only one module called Acquire
that interfaces the USB-6211 card. If the card needs changed in future then only this module needs rewritten. It is the only module in the entire package that is just a function, as spawning processes that are class methods using the multiprocessing
module is very difficult (maybe impossible?). All this module needs are the session settings (for channel names and rates) and a multiprocessing.Queue
to report the data back to DataManager
.
mallard.gui
This provides the graphical user interface using wxPython. MFrame
is the base wx.Frame
used, which owns a CaptureNotebook
. This CaptureNotebook
holds CapturePane
s, each of which corresponds to a CaptureSession
. The GraphManager
provides a link between the DataManager
and the gui, updating the graph on screen with the data coming in. SettingsDialog
simply displays a dialog that lets the user change all the different settings.
mallard.test
Holds all the obsolete and test scripts created along the way, in case they come in handy at a later point. May be changed to `old' in future.
Troubleshooting
Mallard errors
ImportError: Matplotlib backend_wx and backend_wxagg require wxversion, which was not found.
Sometimes the wxversion module is missing. Download it from http://svn.wxwidgets.org/viewvc/wx/wxPython/trunk/wxversion/wxversion.py?view=co and install in your site-packages
directory (for Anaconda this is usually C:/Anaconda/lib/site-packages
).
NIDAQmx Errors
The specified resource is reserved. The operation could not be completed as specified.
This means a task is still running on the device. To end the task and recover, open the NIDAQmx utility from Start -> National Instruments -> NiMAX. Select Devices and Interfaces -> NI USB 6211 then select 'Reset Device', and await confirmation.
Authors
Originally written by Kieran Campbell (@kieranrcampbell) as part of the CERN summer student program.