# Open Music Kontrollers

## Firmware

So we have a lot of sensors and a micro controller. How do we get to our event signals? That is what this page is all about. One component of our event signal for a given magnetic source is its distance from the sensor. We will start right there and firstly look at the relationship of magnetic field and distance.

### Distance - magnetic field - relationship

The strength of the magnetic field of a permanent magnet decreases with increasing distance from the sensor. This decrease is non-linear and the equations involved are non-trivial, it depends on the material and form of the permanent magnet. As a general rule, a cubic relationship is assumed. Nowadays, magnetic fields can be approximated with finite-element calculations, this however is overkill and not at all possible in realtime on a microcontroller. To approximate the distance of a given magnet from the sensor based on the sensed magnetic field flux at the latter in realtime it is therefore best to do some non-linear curve fitting for the given pair of sensor and magnet before-hand. In realtime, the fitted distance-function is simply applied to the sensed magnetic field flux.

Instead of distance, we use the concept of vicinity ($1-distance$ ) because in the end we are interested in how near we are to the sensors and not how far away from them. The following simple function to get normalized vicinity ${y}_{n}$ from the normalized magnetic flux ${B}_{n}$ has been shown to be adequate for the type of sensors and magnets (Neodymium) we use:

$\begin{array}{rcll}{y}_{n}& =& {c}_{0}\cdot \sqrt{{B}_{n}}+{c}_{1}\cdot \sqrt{{B}_{n}}+{c}_{2}\cdot {B}_{n}& \text{(1)}\text{}\text{}\end{array}$ Measured and least-squares estimate between magnetic field at and vicinity to the sensor.

The relationship can vary considerably between different types of permanent magnets, the Chimaera therefore can be calibrated on-the-fly to different types of permanent magnets. This is done with a five point least squares fit and leads to an analytical solution with the help of some linear algebra.

$\begin{array}{rcll}B& =& \left(\begin{array}{ccc}\hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill \sqrt{{b}_{1}}\hfill & \hfill \sqrt{{b}_{1}}\hfill & \hfill {b}_{1}\hfill \\ \hfill \sqrt{{b}_{2}}\hfill & \hfill \sqrt{{b}_{2}}\hfill & \hfill {b}_{2}\hfill \\ \hfill \sqrt{{b}_{3}}\hfill & \hfill \sqrt{{b}_{3}}\hfill & \hfill {b}_{3}\hfill \\ \hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \end{array}\right)& \text{(1)}\text{}\text{}\\ y& =& \left(\begin{array}{c}\hfill 0\hfill \\ \hfill {y}_{1}\hfill \\ \hfill {y}_{2}\hfill \\ \hfill {y}_{3}\hfill \\ \hfill 1\hfill \end{array}\right)& \text{(2)}\text{}\text{}\\ c& =& \left(\begin{array}{c}\hfill {c}_{0}\hfill \\ \hfill {c}_{1}\hfill \\ \hfill {c}_{2}\hfill \end{array}\right)={\left({B}^{T}\cdot B\right)}^{-1}\cdot {B}^{T}\cdot y& \text{(3)}\text{}\text{}\end{array}$

### Sensor calibration

Get familiar first with the hardware of the Chimaera sensor unit, it will help a lot to understand all the following.

Although we now know the relationship between magnetic flux and vicinity we cannot yet approximate the distance of a magnet from its sensor. From the sensor unit we get a voltage which is linearly related to the magnetic flux the currently active sensor is in. However, the sensors do not all have the exactly same sensitivity and quiescent output, their values scatter over a given range. We have to take care of this firstly.

An ideal linear hall-effect sensor outputs a voltage ${V}_{s}$ dependent on its sensitivity $S$ and magnetic flux $B$ relative to its quiescent voltage output ${V}_{q}$ .

$\begin{array}{rcll}{V}_{s}& =& {V}_{q}+S\cdot B& \text{(1)}\text{}\text{}\end{array}$

Both ${V}_{q}$ and $S$ vary between sensors. To make the voltage output of sensors comparable to each other (which is a prerequisite to interpolate between them later on), we need to calibrate them and apply the calibration data in a normalization step. But before we can do that we also have to consider the amplification of the voltage signal at the sensor unit.

The voltage output ${V}_{o}$ after the amplification is dependent on the sensor voltage ${V}_{s}$ , the reference voltage ${V}_{r}$ at the OpAmp, and the amplification factor $A$ of the OpAmp.

$\begin{array}{rcll}A& =& 1+\frac{R{V}_{1}}{{R}_{1}}& \text{(1)}\text{}\text{}\\ {V}_{o}& =& {V}_{r}+\left({V}_{s}-{V}_{r}\right)\cdot A& \text{(2)}\text{}\text{}\\ {V}_{o}& =& {V}_{r}+\left({V}_{q}-{V}_{r}+S\cdot B\right)\cdot A& \text{(3)}\text{}\text{}\end{array}$

${V}_{o}$ is what we measure with the analog-to-digital converters at the microcontroller. This is the only thing we can measure directly. We now have to apply some simple math to get to one value we want in the end: the relative vicinity normalized relative to the whole sensor array.

In the case of no magnetic field, we can measure the amplified quiescent voltage ${V}_{oq}$ of each sensor from which we can solve for ${V}_{q}$ and rewrite ${V}_{o}$ .

$\begin{array}{rcll}B& =& 0& \text{(1)}\text{}\text{}\\ {V}_{s}& =& {V}_{q}& \text{(2)}\text{}\text{}\\ {V}_{oq}& =& {V}_{r}+\left({V}_{q}-{V}_{r}\right)\cdot A& \text{(3)}\text{}\text{}\\ {V}_{q}& =& \frac{{V}_{oq}-{V}_{r}}{A}+{V}_{r}& \text{(4)}\text{}\text{}\\ {V}_{o}& =& {V}_{oq}+A\cdot S\cdot B& \text{(5)}\text{}\text{}\end{array}$

${V}_{o}$ we can measure, ${V}_{oq}$ we can measure, too. $B$ we want to know finally. The next therefore is to decipher $A$ and $S$ . For this we define a new single parameter $AS=A\cdot S$ . With a known $B$ we could already calculate the latter, but we want to calculate it over both polarization domains concurrently to be more robust. We therefore look at the output voltages for an arbitrary magnetic field strength, once south polarized ${B}_{1}$ and once north polarized ${B}_{2}$ .

$\begin{array}{rcll}{B}_{2}& =& -{B}_{1}& \text{(1)}\text{}\text{}\\ {V}_{o1}& =& {V}_{oq}+AS\cdot {B}_{1}& \text{(2)}\text{}\text{}\\ {V}_{o2}& =& {V}_{oq}+AS\cdot {B}_{2}& \text{(3)}\text{}\text{}\\ \frac{{V}_{o1}}{{V}_{o2}}& =& \frac{{V}_{oq}+AS\cdot {B}_{1}}{{V}_{oq}+AS\cdot {B}_{2}}& \text{(4)}\text{}\text{}\\ {V}_{o1}\cdot \left({V}_{oq}+AS\cdot {B}_{2}\right)& =& {V}_{o2}\cdot \left({V}_{oq}+AS\cdot {B}_{1}\right)& \text{(5)}\text{}\text{}\\ {V}_{o1}\cdot {V}_{oq}+{V}_{o1}\cdot AS\cdot {B}_{2}& =& {V}_{o2}\cdot {V}_{oq}+{V}_{o2}\cdot AS\cdot {B}_{1}& \text{(6)}\text{}\text{}\\ AS\cdot \left({V}_{o1}\cdot {B}_{2}-{V}_{o2}\cdot {B}_{1}\right)& =& {V}_{oq}\cdot \left({V}_{o2}-{V}_{o1}\right)& \text{(7)}\text{}\text{}\\ AS& =& \frac{{V}_{oq}\cdot \left({V}_{o2}-{V}_{o1}\right)}{{V}_{o1}\cdot {B}_{2}-{V}_{o2}\cdot {B}_{1}}& \text{(8)}\text{}\text{}\\ AS& =& \frac{{V}_{oq}\cdot \left({V}_{o2}-{V}_{o1}\right)}{|B|\cdot \left(-{V}_{o1}-{V}_{o2}\right)}& \text{(9)}\text{}\text{}\\ AS& =& \frac{{V}_{oq}\cdot \left({V}_{o1}-{V}_{o2}\right)}{|B|\cdot \left({V}_{o1}+{V}_{o2}\right)}& \text{(10)}\text{}\text{}\end{array}$

So, with measured ${V}_{oq}$ at $B=0$ and ${V}_{o1},{V}_{o2}$ at a arbitrary $|B|$ , we get to $AS$ . But how do we know $|B|$ ? It’s quite simple, we can numerically approximate it with the inverse of the magnetic-field-distance relationship, because vicinity is something we can easily measure, too. By doing so, we also have to define what the normalized magnetic flux ${B}_{n}$ actually is.

$\begin{array}{rcll}{y}_{n}& =& {c}_{0}\cdot \sqrt{{B}_{n}}+{c}_{1}\cdot \sqrt{{B}_{n}}+{c}_{2}\cdot {B}_{n}& \text{(1)}\text{}\text{}\\ {B}_{n}& =& {f}^{-1}\left({y}_{n}\right)& \text{(2)}\text{}\text{}\\ {B}_{n}& =& \frac{B-{B}_{min}}{{B}_{max}-{B}_{min}}& \text{(3)}\text{}\text{}\\ B& =& {B}_{n}\cdot \left({B}_{max}-{B}_{min}\right)+{B}_{min}& \text{(4)}\text{}\text{}\end{array}$

Let us get one step further. In the end we want vicinity to be mapped continuously from $\left[0-1\right]$ . $y=0$ will be at a threshold magnetic flux ${B}_{min}$ , $y=1$ in turn at the maximal to expect magnetic flux ${B}_{max}$ . ${B}_{n}$ will be the normalized measured magnetic flux $B$ between the two extremes and depends on the three constants ${V}_{oq}$ , $U$ and $W$ :

$\begin{array}{rcll}B& =& \frac{{V}_{o}-{V}_{oq}}{AS}& \text{(1)}\text{}\text{}\\ B& =& \left({V}_{o}-{V}_{oq}\right)\cdot A{S}^{-1}& \text{(2)}\text{}\text{}\\ N& =& {B}_{max}-{B}_{min}& \text{(3)}\text{}\text{}\\ {B}_{n}& =& \left(B-{B}_{min}\right)\cdot {N}^{-1}& \text{(4)}\text{}\text{}\\ {B}_{n}& =& \left(\left({V}_{o}-{V}_{oq}\right)\cdot A{S}^{-1}-{B}_{min}\right)\cdot {N}^{-1}& \text{(5)}\text{}\text{}\\ {B}_{n}& =& \left({V}_{o}-{V}_{oq}\right)\cdot A{S}^{-1}\cdot {N}^{-1}-{B}_{min}\cdot {N}^{-1}& \text{(6)}\text{}\text{}\\ U& =& A{S}^{-1}\cdot {N}^{-1}& \text{(7)}\text{}\text{}\\ W& =& {B}_{min}\cdot {N}^{-1}& \text{(8)}\text{}\text{}\\ {B}_{n}& =& \left({V}_{o}-{V}_{oq}\right)\cdot U-W& \text{(9)}\text{}\text{}\end{array}$

${V}_{oq}$ and $U$ are different for each sensor (so they are actually two arrays of constants each), $W$ is the same for the whole sensor array. By calibrating them once, we get from the sensor unit voltage output ${V}_{o}$ directly to the normalized magnetic field strength ${B}_{n}$ with only 3 mathematical operations (efficiency is important as we are on a microcontroller, e.g. we have to cope with limited computation power).

From the normalized magnetic flux ${B}_{n}$ we then get to the vicinity $y$ . As those vicinities are comparable over the whole sensor array, we are now ready to interpolate the position $x$ of the magnetic source along the sensor array axis.

### Blob event handling

With a blob we refer to a recognized magnetic field source (e.g. permanent magnet) that is to be tracked over space and time. Blob event handling can be divided into several stages: decipher polarity, narrowing down the area of interest on the sensor array, actual blob detection, interpolation of blob position, group association and blob tracking.

#### Magnetic flux polarity

First thing that is to be done is to discriminate between the polarities of the potentially present magnetic field sources. For each sensor value, we therefore subtract the previously calibrated quiescent value, so we get the difference of the present raw sensor value from its quiescent value. The sign of this difference gives us the polarity of the magnetic source we are encountering here, e.g. south and north polarity. The absolute value of the difference gives us the strength of the magnetic field (which is to be normalized with previously calibrated factors in a later step).

#### Area of interest

After each scan of the whole sensor array, areas of interest are marked on the array. These are regions, where the sensor value exceeds the threshold magnetic flux ${B}_{min}$ . The rest of the sensors are not of interest for the further steps. We do not apply the distance-magnetic-field relationship over all sensors at this step, as it would be computationally too costly. During calibration, for each sensor we precalculated its raw value ${V}_{o,min}$ corresponding to threshold magnetic flux ${B}_{min}$ . This then can be efficiently checked against for each sensor at each update step to find out whether the threshold has been exceeded or not.

#### Blob detection

In the next step, we search for peaks in the areas of interest. A peak will correspond to an individual magnetic field source and is simply defined as a sensor with an absolute value which is greater than the values of a given number of adjacent sensors in the same area of interest. We save all found peaks along the sensor array for the blob’s positional interpolation in the next step.

#### Blob interpolation

As always more than one sensor is excited by an individual magnetic source, we can decipher its real position by interpolation around the peak sensor. We therefore fit curves ${y}_{n}\left(x\right)$ over a varying number of sensor values around the peak and find the maximum $\left(X,Y\right)$ by finding the cross point of the curve’s derivative. Only at this step we convert the sensor values into normalized vicinities based on calibration data. For efficiency reasons, we use a precalculated lookup table for the distance-magnetic-field relationship, because the cube root and square root functions would be too costly.

$\begin{array}{rcll}{y}_{n}\left(X\right)& =& Y& \text{(1)}\text{}\text{}\\ \frac{\partial {y}_{n}\left(X\right)}{\partial x}& =& 0& \text{(2)}\text{}\text{}\end{array}$

We can use varying numbers of adjacent sensors to the peak and fit linear, quadratic, cubic or spline functions through the values. The coordinates $\left(X,Y\right)$ of the maximum will be the current position of the blob on the sensor array.

$\begin{array}{rcll}{y}_{n}\left(x\right)& =& A\cdot x+B& \text{(1)}\text{}\text{}\\ {y}_{n}\left(x\right)& =& A\cdot {x}^{2}+B\cdot x+C& \text{(2)}\text{}\text{}\\ {y}_{n}\left(x\right)& =& A\cdot {x}^{3}+B\cdot {x}^{2}+C\cdot x+D& \text{(3)}\text{}\text{}\end{array}$ Vicinities of four adjacent sensors and no fit (order 0), e.g. the highest value is its own fit. Vicinities of four adjacent sensors and a linear fit (order 1) between the two highest values. Vicinities of four adjacent sensors and a quadratic, aka hyperbolic fit (order 2) betwwen the three highest values. Vicinities of four adjacent sensors and a cubic fit with Catmull-Rom splines (order 3) between all four values. Only the spline segment between the highest values is shown. Vicinities of four adjacent sensors and a fit with a Lagrange polynomial (order 3) between all four values.

#### Group association

For convenience, we have introduced a concept of groups. Each blob has not only a given position on the sensor array, but is also affiliated to a group. A group is defined by a range (${x}_{min}..{x}_{max}$ ) on the sensor array and magnetic field flux polarity (south, north or both). Only blobs with a position in the range of - and a polarity corresponding to a given group will be associated with the latter.

Groups can overlap, e.g. there can be two groups for the same range with differing polarities. A blob can only be part of one group at a time though and will be affiliated to the first matching one.

#### Blob tracking

Up to now, everything we have done was based on the current state of the sensor array. However, to track blobs over time, there needs to be a means to associate identified blobs in the current iteration to blobs found in previous iterations to decipher whether new blobs have appeared or old ones have disappeared.

To link current blobs to previous blobs they are matched by position on the sensor array. Each new blob gets a unique ID, which will persist over time as long as the blob can be linked to itself in a previous iteration step, until the blob disappears. In the special case when a blob can be linked but has exited its previous group, it will be regarded as a new one, too.

Out of the blob tracking, we get a stream of event signals: When a new blob has appeared, an on event is triggered. When a blob could be linked to a previous iteration step, a set event is triggered to update the blobs position. When a blob has disappeared, an off event is triggered. All event signals will be triggered with complete set of current parameters: position $\left(X,Y\right)$ , polarity (south or north) and group association.

### Summary

So, what the firmware does in the end is: converting a raw array of sampled sensor values into a stream of blob event signals. The event signals now can be translated into a given transport protocol and sent over the network to drive other software or hardware. This is elaborated upon in a dedicated section, the usage.

Last update - 15 Jul 2019