Fast data capture with the Raspberry Pi

Video signal captured at 2.6 megasamples per second

Adding an Analog-to-Digital Converter (ADC) to the Raspberry Pi isn’t difficult, and there is ample support for reading a single voltage value, but what about getting a block of samples, in order to generate an oscilloscope-like trace, as shown above?

By careful manipulation of the Linux environment, it is possible to read the voltage samples in at a decent rate, but the big problem is the timing of the readings; the CPU is frequently distracted by other high-priority tasks, so there is a lot of jitter in the timing, which makes the analysis & display of the waveforms a lot more difficult – even a fast board such as the RPi 4 can suffer from this problem.

We need a way of grabbing the data samples at regular intervals without any CPU intervention; that means using Direct Memory Access, which operates completely independently of the processor, so even the cheapest Pi Zero board delivers rock-solid sample timing.

Direct Memory Access

Direct Memory Access (DMA) can be set up to transfer data between memory and peripherals, without any CPU intervention. It is a very powerful technique, and as a result, can easily cause havoc if programmed incorrectly. I strongly recommend you read my previous post on the subject, which includes some simple demonstrations of DMA in action, but here is a simplified summary:

  1. The CPU has three memory spaces: virtual, bus and physical. DMA accesses use bus memory addresses, but a user program employs virtual addresses, so it is necessary to translate between the two.
  2. When writing to memory, the CPU is actually writing to an on-chip cache, and sometime later the data is written to main memory. If the DMA controller tries to fetch the data before the cache has been emptied, it will get incorrect values. So it is necessary for all DMA data to be in uncached memory.
  3. If compiler optimisation is enabled, it can bypass some memory read operations, giving a false picture of what is actually in memory. The qualifier ‘volatile’ might be needed to make sure that variables changed by DMA are correctly read by the processor.
  4. The DMA controller receives its next instruction via a Control Block (CB) which specifies the source & destination addresses, and the number of bytes to be transferred. Control Blocks can be chained, so as to create a sequence of actions.
  5. DMA transactions are normally triggered by a data request from a peripheral, otherwise they run through at full speed without stopping.
  6. If the DMA controller receives incorrect data, it can overwrite any area of memory, or any peripheral, without warning. This can cause unusual malfunctions, system crashes or file corruption, so care is needed.

For this project, I’ve abstracted the DMA and I/O functions into the new files rpi_dma_utils.c and rpi_dma_utils.h. The handling of the memory spaces has also been improved, with a single structure for each peripheral or memory area:

// Structure for mapped peripheral or memory
typedef struct {
    int fd,         // File descriptor
        h,          // Memory handle
        size;       // Memory size
    void *bus,      // Bus address
        *virt,      // Virtual address
        *phys;      // Physical address
} MEM_MAP;

To access a peripheral, the structure is initialised with the physical address:

#define SPI0_BASE       (PHYS_REG_BASE + 0x204000)

// Use mmap to obtain virtual address, given physical
void *map_periph(MEM_MAP *mp, void *phys, int size)
{
    mp->phys = phys;
    mp->size = PAGE_ROUNDUP(size);
    mp->bus = phys - PHYS_REG_BASE + BUS_REG_BASE;
    mp->virt = map_segment(phys, mp->size);
    return(mp->virt);
}

MEM_MAP spi_regs;
map_periph(&spi_regs, (void *)SPI0_BASE, PAGE_SIZE);

Then a macro is used to access a specific register:

#define REG32(m, x) ((volatile uint32_t *)((uint32_t)(m.virt)+(uint32_t)(x)))
#define SPI_DLEN        0x0c

*REG32(spi_regs, SPI_DLEN) = 0;

The advantage of this approach is that it is easy to set or clear individual bits within a register, e.g.

*REG32(spi_regs, SPI_CS) |= 1;

Note that the REG32 macro uses the ‘volatile’ qualifier to ensure that the register access will still be executed if compiler optimisation is enabled.

Analog-to-Digital Converters (ADCs)

There are 3 ways an ADC can be linked to the Raspberry Pi (RPi):

  1. Inter-Integrated Circuit (I2C) serial bus
  2. Serial Peripheral Interface (SPI) serial bus
  3. Parallel bus

The I2C interface is the simplest from a hardware point of view, since it only has 2 connections: clock and data. However, these devices tend to be a bit slow, and the RPi I2C interface doesn’t support DMA, so we won’t be using this method.

The parallel interface is the most complicated, as it has 1 wire for each data bit, plus one or more clock lines: I’ll be looking at this in a future blog post.

This leaves the SPI interface, which is a good compromise between complexity and speed; it has only 4 connections (clock, data out, data in and chip select) but is capable of achieving over 1 megasample per second.

In this post we’ll be using 2 SPI ADC chips; the Microchip MCP3008 which is specified as 100 Ksamples/sec maximum (though I’ve only achieved 80 KS/s, for reasons I’ll discuss later), and the Texas Instruments ADS7884 which can theoretically achieve 3 Msample/s; I’ve run that at 2.6 MS/s. Both chips are 10-bit, so return a value of 0 to 1023, when measuring 0 to 3.3 volts.

MCP3008

The RasPiO Analog Zero board ( https://rasp.io/analogzero/ ) has the Microchip MCP3008 ADC on it, and very little else.

It is in the same form-factor as the RPi Zero, but I used a version 3 CPU board for most of my testing. There are 8 analogue input channels, but only a single ADC, that has to be switched to the appropriate channel prior to conversion. The voltage reference is taken from the RPi 3.3 volt rail; if you need greater stability & accuracy, a standalone voltage reference can be used instead.

SPI interface

The board is tied to the SPI0 interface on the RPi, using 4 connections

  • GPIO8 CE0: SPI 0 Chip Enable 0
  • GPIO11 SCLK: Clock signal
  • GPIO10 MOSI: data output to ADC
  • GPIO9 MISO: data input from ADC

The Chip Enable (or Chip Select as it is often known) is used to frame the overall transfer; it is normally high, then is set low to start the analog-to-digital conversion, and is held low while the data is transferred to & from the device.

Getting a single sample from the ADC is really easy in Python:

from gpiozero import MCP3008
adc = MCP3008(channel=0)
print(adc.value * 3.3)
adc.close()

We’ll be diving a bit deeper into the way the SPI interface works, so here is the same operation in Python, but direct-driving the SPI interface:

import spidev
spi = spidev.SpiDev()
spi.open(0, 0)
spi.max_speed_hz = 500000
spi.mode = 0
msg = [0x01,0x80,0x00]
rsp = spi.xfer2(msg)
val = ((rsp[1]*256 + rsp[2]) & 0x3ff) * 3.3 / 1.024
print(val)

The most useful diagnostic method is to view the signals on an oscilloscope, so here are the corresponding traces; the scale is 20 microseconds per division (per square) horizontally, and 5 volts per division vertically:

RPi SPI access of an MCP3008 ADC

You can see the Chip Select frames the transaction, but remains active (low) for about 120 microseconds after the transfer is finished; that is something we’ll need to improve to get better speeds. The clock is 50 kHz as specified in the code, but this can be up to 2 MHz. The MOSI (CPU output) data is as specified in the data sheet, a value of 01 80 hex has a ‘1’ start bit, followed by another ‘1’ to select single-ended mode (not differential). MISO (CPU input) data reflects the voltage value measured by the ADC. The data is always sent most-significant-bit first, and the first return byte is ignored (since the ADC hadn’t started the conversion), so the second byte has to be multiplied by 256, and added to the third byte.

You’ll see there is a downward curve at the end of the MISO trace; this shows that the line isn’t being driven high or low, and is floating. It is worth watching out for signals like this, since they can cause problems as they drift between 1 and 0; in this case the transition is harmless as the transfer cycle has already finished.

MCP3008 software

Here is the C equivalent of the Python code:

// Set / clear SPI chip select
void spi_cs(int set)
{
    uint32_t csval = *REG32(spi_regs, SPI_CS);

    *REG32(spi_regs, SPI_CS) = set ? csval | 0x80 : csval & ~0x80;
}

// Transfer SPI bytes
void spi_xfer(uint8_t *txd, uint8_t *rxd, int len)
{
    while (len--)
    {
        *REG8(spi_regs, SPI_FIFO) = *txd++;
        while((*REG32(spi_regs, SPI_CS) & (1<<17)) == 0) ;
        *rxd++ = *REG32(spi_regs, SPI_FIFO);
    }
}

// Fetch single 10-bit sample from ADC
int adc_get_sample(int chan)
{

    uint8_t txdata[3]={0x01,0x80|(chan<<4),0}, rxdata[3];
    
    spi_cs(1);
    spi_xfer(txdata, rxdata, sizeof(txdata));
    spi_cs(0);
    return(((rxdata[1]<<8) | rxdata[2]) & 0x3ff);
}

This takes 3 bytes to transfer 10 data bits, which is a bit wasteful. It is worth reading the MCP3008 data sheet, which explains that the leading ‘1’ of the outgoing data is used to trigger the conversion, so the whole cycle can be compressed into 16 bits, if you ignore the last data bit:

// Fetch 9-bit sample from ADC
int adc_get_sample(int chan)
{
    uint8_t txdata[2]={0xc0|(chan<<3),0}, rxdata[2];
    
    spi_cs(1);
    spi_xfer(txdata, rxdata, sizeof(txdata));
    spi_cs(0);
    return((((int)rxdata[0] << 9) | ((int)rxdata[1] << 1)) & 0x3ff);
}

You’ll see that the transmit bytes 0x01,0x80 have been shifted left by 7 bits to make one byte 0xc0, and this results in the response data being shifted left by the same amount.

A single transfer can easily be done using DMA, since the SPI controller has an auto-chip-select mode that handles the CE signal for us:

// Fetch single sample from MCP3008 ADC using DMA
int adc_dma_sample_mcp3008(MEM_MAP *mp, int chan)
{
    DMA_CB *cbs=mp->virt;
    uint32_t dlen, *txd=(uint32_t *)(cbs+2);
    uint8_t *rxdata=(uint8_t *)(txd+0x10);

    enable_dma(DMA_CHAN_A);
    enable_dma(DMA_CHAN_B);
    dlen = 4;
    txd[0] = (dlen << 16) | SPI_TFR_ACT;
    mcp3008_tx_data(&txd[1], chan);
    cbs[0].ti = DMA_SRCE_DREQ | (DMA_SPI_RX_DREQ << 16) | DMA_WAIT_RESP | DMA_CB_DEST_INC;
    cbs[0].tfr_len = dlen;
    cbs[0].srce_ad = REG_BUS_ADDR(spi_regs, SPI_FIFO);
    cbs[0].dest_ad = MEM_BUS_ADDR(mp, rxdata);
    cbs[1].ti = DMA_DEST_DREQ | (DMA_SPI_TX_DREQ << 16) | DMA_WAIT_RESP | DMA_CB_SRCE_INC;
    cbs[1].tfr_len = dlen + 4;
    cbs[1].srce_ad = MEM_BUS_ADDR(mp, txd);
    cbs[1].dest_ad = REG_BUS_ADDR(spi_regs, SPI_FIFO);
    *REG32(spi_regs, SPI_DC) = (8 << 24) | (4 << 16) | (8 << 8) | 4;
    *REG32(spi_regs, SPI_CS) = SPI_TFR_ACT | SPI_DMA_EN | SPI_AUTO_CS | SPI_FIFO_CLR | SPI_CSVAL;
    *REG32(spi_regs, SPI_DLEN) = 0;
    start_dma(mp, DMA_CHAN_A, &cbs[0], 0);
    start_dma(mp, DMA_CHAN_B, &cbs[1], 0);
    dma_wait(DMA_CHAN_A);
    return(mcp3008_rx_value(rxdata));
}

// Return Tx data for MCP3008
int mcp3008_tx_data(void *buff, int chan)
{
    uint8_t txd[3]={0x01, 0x80|(chan<<4), 0x00};
    memcpy(buff, txd, sizeof(txd));
    return(sizeof(txd));
}

// Return value from ADC Rx data
int mcp3008_rx_value(void *buff)
{
    uint8_t *rxd=buff;
    return(((int)(rxd[1]&3)<<8) | rxd[2]);
}

When testing new DMA code, it is not unusual for there to be an error such that the DMA cycle never completes, so the dma_wait function has a timeout:

// Wait until DMA is complete
void dma_wait(int chan)
{
    int n = 1000;

    do {
        usleep(100);
    } while (dma_transfer_len(chan) && --n);
    if (n == 0)
        printf("DMA transfer timeout\n");
}

So we have code to do a single transfer, can’t we use the same idea to grab multiple samples in one transfer? The problem is the CS line; this has to be toggled for each value, and the auto-chip-select mode only works for a single transfer; despite a lot of experimentation, I couldn’t find any way of getting the SPI controller to pulse CS low for each ADC cycle in a multi-cycle capture.

The solution to this problem comes in treating the transmit and receive DMA operations very differently. The receive operation simply keeps copying the 32-bit data from the SPI FIFO into memory, until all the required data has been captured. In contrast, the transmit side is repeatedly sending the same trigger message to the ADC (0x01, 0x80, 0x00 in the above example). Since the same message is repeating, we could set up a small sequence of DMA Control Blocks (CBs):

CB1: set chip select high
CB2: set chip select low
CB3: write next 32-bit word to the FIFO

The controller is normally executing CB3, waiting for the next SPI data request. When this arrives, it executes CB1 then CB2, briefly setting the chip select high & low to start a new data capture. It then stops in CB3 again, waiting for the next data request. Using this method, the typical width of the CS high pulse is 330 nanoseconds, which is more than adequate to trigger the ADC.

The bulk of code is the same as the previous example, here are the control block definitions:

    // Control block 0: read data from SPI FIFO
    cbs[0].ti = DMA_SRCE_DREQ | (DMA_SPI_RX_DREQ << 16) | DMA_WAIT_RESP | DMA_CB_DEST_INC;
    cbs[0].tfr_len = dlen;
    cbs[0].srce_ad = REG_BUS_ADDR(spi_regs, SPI_FIFO);
    cbs[0].dest_ad = MEM_BUS_ADDR(mp, rxdata);
    // Control block 1: CS high
    cbs[1].srce_ad = cbs[2].srce_ad = MEM_BUS_ADDR(mp, pindata);
    cbs[1].dest_ad = REG_BUS_ADDR(gpio_regs, GPIO_SET0);
    cbs[1].tfr_len = cbs[2].tfr_len = cbs[3].tfr_len = 4;
    cbs[1].ti = cbs[2].ti = DMA_DEST_DREQ | (DMA_SPI_TX_DREQ << 16) | DMA_WAIT_RESP | DMA_CB_SRCE_INC;
    // Control block 2: CS low
    cbs[2].dest_ad = REG_BUS_ADDR(gpio_regs, GPIO_CLR0);
    // Control block 3: write data to Tx FIFO
    cbs[3].ti = DMA_DEST_DREQ | (DMA_SPI_TX_DREQ << 16) | DMA_WAIT_RESP | DMA_CB_SRCE_INC;
    cbs[3].srce_ad = MEM_BUS_ADDR(mp, &txd[1]);
    cbs[3].dest_ad = REG_BUS_ADDR(spi_regs, SPI_FIFO);
    // Link CB1, CB2 and CB3 in endless loop
    cbs[1].next_cb = MEM_BUS_ADDR(mp, &cbs[2]);
    cbs[2].next_cb = MEM_BUS_ADDR(mp, &cbs[3]);
    cbs[3].next_cb = MEM_BUS_ADDR(mp, &cbs[1]);

A disadvantage of this approach is that we’re transferring 32 bits in order to get 10 bits of ADC data, which is quite wasteful; if the DMA controller could be persuaded to transfer 16 bits at a time, we’d be able to double the speed, but all my attempts to do this have failed.

However, on the positive side, it does produce an accurately-timed data capture with no CPU intervention:

Raspberry Pi MCP3008 ADC input using DMA

The oscilloscope trace just shows 4 transfers, but the technique works just as well with larger data blocks; here is a trace of 500 samples at 80 Ksample/s

To be honest, the ADC was overclocked to achieve this sample rate; the data sheet implies that the maximum SPI clock should be around 2 MHz with a 3.3V supply voltage, and the actual value I’ve used is 2.55 MHz, so don’t be surprised if this doesn’t work reliably in a different setup.

ADS7884

In the title of this blog post I promised ‘fast’ data capture, and I don’t think 80 Ksample/s really qualifies as fast; the generally accepted definition is at least 10 Msample/s, but that would require an SPI clock over 100MHz, which is quite unrealistic.

The ADS7884 is a fast single-channel SPI ADC; it can acquire 3 Msample/s, with an SPI clock of 48 MHz, but you do have to be quite careful when dealing with signals this fast; a small amount of stray capacitance or inductance can easily distort the signals so that the transfers are unreliable. All connections must be kept short, especially the clock, power and ground, which ideally should be less than 50 mm (2 inches) long.

The ADC chip is in a very small 6-pin package (0.95 mm pin spacing) so I soldered it to a Dual-In-Line (DIL) adaptor, with 1 uF and 10 nF decoupling capacitors as close to the power & ground pins as possible. This arrangement is then mounted on a solder prototyping board (not stripboard) with very short wires soldered to the RPi I/O connector.

ADS7884 on a prototyping board

You may think that the ADC should still work correctly in a poor layout, if the clock frequency is reduced. This may not be true as, generally speaking, the faster the device, the more sensitive it is to the quality of the external signals. If they aren’t clean enough, the ADC will still malfunction, no matter how slow the clock is.

The device pins are:

1  Supply (3.3V)
2  Ground
3  VIN (voltage to be measured)
4  SCLK (SPI clock)
5  SDO (SPI data output)
6  CS (chip select, active low)

You’ll see that there is no data input line; this is because, unlike the MCP3008, there is nothing to control; just set CS low, toggle the clock 16 times, then set CS high, and you’ll have the data.

This can be demonstrated by a Python program:

import spidev
bus, device = 0, 0
spi = spidev.SpiDev()
spi.open(bus, device)
spi.max_speed_hz = 1000000
spi.mode = 0
msg = [0x00,0x00]
spi.xfer2(msg)
res = spi.xfer2(msg)
val = (res[0] * 256 + res[1]) >> 6
print("%1.3f" % val * 3.3 / 1024.0)

You’ll see that I’ve discarded the first sample from the ADC; that is because it always returns the data from the previous sample, i.e. it outputs the last sample while obtaining the next.

When creating the DMA software, it is tempting to use the same technique I employed on the MCP3008, but I want really fast sampling, and using a 32-bit word to carry 10 bits of data seems much too wasteful.

Since the SPI transmit line is unused (as the ADS7884 doesn’t have a data input) we can use it for another purpose, so why not use it to drive the chip select line? This means we can drive CS high or low whenever we want, just by setting the transmit data.

So the connections between the ADC and RPi are:

Pin 1: 3.3V supply 
Pin 2: ground 
Pin 3: voltage to be measured
Pin 4: SPI0 clock, GPIO11
Pin 5: SPI0 MISO,  GPIO9
Pin 6: SPI0 MOSI,  GPIO10 (ADC chip select)

If you are driving other SPI devices, the absence of a proper chip select could be a major problem. The solution would be to invert the transmitted data, add a NAND gate between the MOSI line and the ADC chip select, and drive the other NAND input with a spare I/O line, to enable (when high) or disable (when low) the ADC transfers. You’d just need to keep an eye on the additional delay in the CS line, which could alter the phase shift between the transmitted and received data.

ADS7884 software

Driving the chip-select line from the SPI data output makes the software quite a bit simpler, just repeat the same 16-bit pattern on the transmit side, and save the received data in a buffer. This is the code:

// Fetch samples from ADS7884 ADC using DMA
int adc_dma_samples_ads7884(MEM_MAP *mp, int chan, uint16_t *buff, int nsamp)
{
    DMA_CB *cbs=mp->virt;
    uint32_t i, dlen, shift, *txd=(uint32_t *)(cbs+3);
    uint8_t *rxdata=(uint8_t *)(txd+0x10);

    enable_dma(DMA_CHAN_A); // Enable DMA channels
    enable_dma(DMA_CHAN_B);
    dlen = (nsamp+3) * 2;   // 2 bytes/sample, plus 3 dummy samples
    // Control block 0: store Rx data in buffer
    cbs[0].ti = DMA_SRCE_DREQ | (DMA_SPI_RX_DREQ << 16) | DMA_WAIT_RESP | DMA_CB_DEST_INC;
    cbs[0].tfr_len = dlen;
    cbs[0].srce_ad = REG_BUS_ADDR(spi_regs, SPI_FIFO);
    cbs[0].dest_ad = MEM_BUS_ADDR(mp, rxdata);
    // Control block 1: continuously repeat last Tx word (pulse CS low)
    cbs[1].srce_ad = MEM_BUS_ADDR(mp, &txd[2]);
    cbs[1].dest_ad = REG_BUS_ADDR(spi_regs, SPI_FIFO);
    cbs[1].tfr_len = 4;
    cbs[1].ti = DMA_DEST_DREQ | (DMA_SPI_TX_DREQ << 16) | DMA_WAIT_RESP | DMA_CB_SRCE_INC;
    cbs[1].next_cb = MEM_BUS_ADDR(mp, &cbs[1]);
    // Control block 2: send first 2 Tx words, then switch to CB1 for the rest
    cbs[2].srce_ad = MEM_BUS_ADDR(mp, &txd[0]);
    cbs[2].dest_ad = REG_BUS_ADDR(spi_regs, SPI_FIFO);
    cbs[2].tfr_len = 8;
    cbs[2].ti = DMA_DEST_DREQ | (DMA_SPI_TX_DREQ << 16) | DMA_WAIT_RESP | DMA_CB_SRCE_INC;
    cbs[2].next_cb = MEM_BUS_ADDR(mp, &cbs[1]);
    // DMA request every 4 bytes, panic if 8 bytes
    *REG32(spi_regs, SPI_DC) = (8 << 24) | (4 << 16) | (8 << 8) | 4;
    // Clear SPI length register and Tx & Rx FIFOs, enable DMA
    *REG32(spi_regs, SPI_DLEN) = 0;
    *REG32(spi_regs, SPI_CS) = SPI_TFR_ACT | SPI_DMA_EN | SPI_AUTO_CS | SPI_FIFO_CLR | SPI_CSVAL;
    // Data to be transmited: 32-bit words, MS bit of LS byte is sent first
    txd[0] = (dlen << 16) | SPI_TFR_ACT;// SPI config: data len & TI setting
    txd[1] = 0xffffffff;                // Set CS high
    txd[2] = 0x01000100;                // Pulse CS low
    // Enable DMA, wait until complete
    start_dma(mp, DMA_CHAN_A, &cbs[0], 0);
    start_dma(mp, DMA_CHAN_B, &cbs[2], 0);
    dma_wait(DMA_CHAN_A);
    // Check whether Rx data has 1 bit delay with respect to Tx
    shift = rxdata[4] & 0x80 ? 3 : 4;
    // Convert raw data to 16-bit unsigned values, ignoring first 3
    for (i=0; i<nsamp; i++)
        buff[i] = ((rxdata[i*2+6]<<8 | rxdata[i*2+7]) >> shift) & 0x3ff;
    return(nsamp);
}

There are a few points that need clarification:

  1. When using DMA, the first word sent to the SPI controller isn’t the data to be transmitted; it is a configuration word that sets the SPI data length, and other parameters. In the MCP3008 implementation I sent it by direct-writing to the FIFO before DMA starts, but at high speed this can cause occasional glitches. So I send the initial SPI configuration using DMA Control Block 2; once that is sent, CB1 performs the main data output.
  2. The phase relationship between the outgoing (chip-select) data and the incoming (ADC value) data isn’t immediately obvious, and as the sampling rate gets faster, this phase relationship changes by 1 bit. To detect this, I first send an all-ones word to keep CS high, then set it low, and check which bit goes low in the received data. This is also done in control block 2, and when that is complete, control block 1 takes over for the remaining transmissions.
  3. The data decoder shifts the raw data depending on the detected phase value, then saves it as 16-bit values in the output array (which has been created in virtual memory using a conventional memory allocation call).
  4. The ADC always returns the result of the previous conversion, so the first sample has to be discarded. Also, the chip select (SPI output) defaults to being low, so the first conversion is usually spurious, and the phase-detection method mentioned above also results in incorrect data. So it is necessary to discard the first 3 samples.

Here is an oscilloscope trace when running at 2.6 megasample/s:

Running the code

The software is in 3 files on Github here.

rpi_adc_dma_test.c
rpi_dma_utils.c
rpi_dma_utils.h

The definition at the top of rpi_adc_dma_test.c needs to be edited to select the ADC (MCP3008 or ADS7884), also rpi_dma_utils.h must be changed to reflect the CPU board you are using (RPi 0/1, 2/3, or 4) and the master clock frequency that will used to determine the SPI clock. Bizarrely, the RPi zero has a 400 MHz master clock, while the later boards use 250 MHz. If you neglect to make this change when using the Pi Zero, the SPI interface will run 1.6 times too fast; I once made this mistake, and to my surprise the ADC still seemed to work fine, even though the resulting 5.76 MS/s data rate is way beyond the values in the ADC data sheet. So if you are an overclocking enthusiast, there is plenty of scope for experimentation.

The code is compiled on the Rasberry Pi using gcc, then run with root privileges using ‘sudo’:

gcc -Wall -o rpi_adc_dma_test rpi_adc_dma_test.c rpi_dma_utils.c
sudo ./rpi_adc_dma_test

The usual security warnings apply when running code with root privileges; the operating system won’t protect you against any undesired operations.

The response will depend on which ADC and processor is in use, but should show the current ADC input value, and the corresponding voltage. This is the Pi Zero:

SPI ADC test v0.03
VC mem handle 5, phys 0xde510000, virt 0xb6f00000
SPI frequency 160000 Hz, 10000 sample/s
ADC value 212 = 0.683V
Closing

There are 2 command-line parameters:

-r to set sample rate        e.g. -r 100000 to set 100 Ksample/s
-n to set number of samples  e.g. -n 500 to fetch 500 samples.

The software reports the actual sample rate; on Pi 3 & 4 boards it generally won’t be the same as the requested value, due to the awkward divisor values to scale down 250 MHz into a suitable SPI clock.

There will be a limit as to how many samples can be gathered, as the raw data is stored in uncached memory. This limit can be increased by allocating more of the RAM to the graphics processor, see the gpu_mem option in config.txt. Alternatively, you could change the code to use cached memory (obtained with mmap) for the raw data buffer, and accept that there will be a delay while the CPU cache is emptied into it.

The output is just a list of voltages, with one sample per line; this can conveniently be piped to a CSV file for plotting in a spreadsheet, for example:

sudo ./rpi_adc_dma_test -r 3000000 -n 500 > test1.csv

The graphs in this post were actually produced using gnuplot, running on the RPi. It is easy to install using ‘sudo apt install gnuplot’, and here is a sample command line, with the graph it produces; I’ve split the commands into multiple lines for clarity:

gnuplot -e "set term png size 420,240 font 'sans,8'; \
  set title '2.5 Msample/s'; set grid; set key noautotitle; \
  set output 'test1.png'; plot 'test1.csv' every ::4 with lines"
Data display using gnuplot

This capture (of a composite video signal) was done on a Pi ZeroW, proving that you don’t need an expensive processor to perform fast & accurate data acquisition.

Copyright (c) Jeremy P Bentham 2020. Please credit this blog if you use the information or software in it.

Raspberry Pi DMA programming in C

If you need a fast efficient way of moving data around a Raspberry Pi system, Direct Memory Access (DMA) is the preferred option; it works independently of the main processor, doing memory and I/O transfers at high speed.

Programming DMA under Linux can be quite difficult; a device driver is normally used, which needs to be custom-written for a specific application. There are also some Raspberry Pi user-mode programs on the Web that can be run from the command line, but they do need to bypass all the usual memory protections, so require root privileges (e.g. run using ‘sudo’). This means that a minor error in the code can cause random corruption of the processor’s memory, resulting in system instability or a crash.

I couldn’t find any simple explanations and code examples on the Web, so decided to write this blog, documenting all the potential problem areas, with fully commented example code.

I’ll be making extensive use of the Broadcom ‘BCM2835 ARM Peripherals’ document, you can get a copy here. There is also an errata document that is worth reading here.

Address spaces

When creating an executable program, you are (possibly unknowingly) using a ‘virtual’ memory space. The addresses you use are just a temporary fiction, created by the Operating System (OS) for the duration of that program. This allows the OS to make maximum usage of the available RAM; when it gets really crowded, your program may even be pushed out to a ‘swap file’ on disk, so it isn’t even in RAM at all.

This is fine for most user programs, but the DMA controller is a relatively simple piece of hardware, so can not handle the free-for-all nature of virtual memory. It requires everything to be at a known address location, in a memory space known as ‘bus memory’. You may already be familiar with this if you have browsed the BCM2835 document; it describes all the peripherals in terms of their bus addresses.

Accessing peripherals

Raspberry Pi peripheral addressing

Peripherals need to be accessible by the DMA controller (for data transfers) and the user program (for initialisation and configuration). It is easy for the DMA controller to access any peripheral; it just uses the bus address, as given in the documentation. However, the user program runs in its own virtual world, so usually can’t access any peripherals, except through device drivers. To gain direct read/write access, it has to specifically request permission from the OS, by making a call to ‘mmap’ with the physical address of the peripheral we want to access:

// Get virtual memory segment for peripheral regs or physical mem
void *map_segment(void *addr, int size)
{
    int fd;
    void *mem;

    if ((fd = open ("/dev/mem", O_RDWR|O_SYNC|O_CLOEXEC)) < 0)
        FAIL("Error: can't open /dev/mem, run using sudo\n");
    mem = mmap(0, size, PROT_WRITE|PROT_READ, MAP_SHARED, fd, (uint32_t)addr);
    close(fd);
    return(mem);
}

The procedure is slightly strange, in that you have to give the function a file descriptor for /dev/mem, and this requires root privileges, but on reflection this isn’t surprising, since we could do a lot of damage by making unauthorised access to the peripherals, so the OS needs to know we have the authority to do this. There is another descriptor, namely /dev/iomem, that doesn’t require root privileges, but that is confined to the GPIO pins, so we can’t use it for DMA.

The mmap function takes a physical address of the peripheral, and opens a window in virtual memory that our program can access; any read or write to the window is automatically redirected to the peripheral.

I’ve said the mmap function needs a physical address, and you may think this is the same as the bus address, but sadly that isn’t true; there are a total of 3 address spaces: bus, physical and virtual. The conversion between bus & physical is quite easy, but changes depending on the Pi board version: this is the code for Pi 2 or 3, with an example of user-mode GPIO access:

#define PHYS_REG_BASE    0x3F000000
#define GPIO_BASE       (PHYS_REG_BASE + 0x200000)
#define PAGE_SIZE       0x1000

void *virt_gpio_regs
virt_gpio_regs = map_segment((void *)GPIO_BASE, PAGE_SIZE);

#define VIRT_GPIO_REG(a) ((uint32_t *)((uint32_t)virt_gpio_regs + (a)))
#define GPIO_LEV0       0x34

// Get an I/P pin value
uint8_t gpio_in(int pin)
{
    uint32_t *reg = VIRT_GPIO_REG(GPIO_LEV0) + pin/32;
    return (((*reg) >> (pin % 32)) & 1);
}

Accessing memory

Raspberry Pi memory addressing

Memory accesses by the DMA controller are a more complicated, as a known fixed address is required. This can be done by mmap; if it is given a zero address, it will allocate a block of memory, and return a virtual pointer to that block:

#define MMAP_FLAGS (MAP_SHARED|MAP_ANONYMOUS|MAP_NORESERVE|MAP_LOCKED)
mem = mmap(0, size, MMAP_FLAGS, fd, 0);

We now have a virtual memory address, which is fine for our user code to access, but can’t be used by the DMA controller, so we need to look up the physical address by consulting the mapping table:

// Return physical address of virtual memory
void *phys_mem(void *virt)
{
    uint64_t pageInfo;
    int file = open("/proc/self/pagemap", 'r');
    
    if (lseek(file, (((size_t)virt)/PAGE_SIZE)*8, SEEK_SET) != (size_t)virt>>9)
        printf("Error: can't find page map for %p\n", virt);
    read(file, &pageInfo, 8);
    close(file);
    return((void*)(size_t)((pageInfo*PAGE_SIZE)));
}

This physical address can be converted to a bus address, and given to the DMA controller, but you will find the end result is quite unreliable; there is a disconnect between the data that the user program is writing, and the values that the DMA controller is reading; the two don’t match up, unless you include very significant delays in the code. This is due to the CPU caching memory accesses.

Memory caching

Raspberry Pi cache areas

Caches are used to temporarily store data values within the CPU, so they can be accessed much faster than main memory. Normally they are completely transparent to the software; the CPU manipulates the cached value of a variable, then the value is written out to main memory after a suitable delay. The length of this delay is dependant on the CPU workload, but may be around 1 second.

This is a major problem when working with DMA; it fetches data and descriptors directly from memory, but if that data was prepared less than a second ago, it may only be in the CPU cache; the memory will still have random values from a previous program, making the DMA controller behave in a totally unpredictable way.

This has the potential to be very nasty problem, since it will come & go depending on the CPU workload and other programs, so can be really difficult to diagnose. We must be absolutely sure that all the cached data has been written to memory before starting DMA. There are various ways this can be done in theory, for example there is a GCC command:

void __clear_cache(void *start, void *end)

however this seems to be more applicable to instruction than data caches, and I didn’t have any success using it.

Another approach is to use the aliases in bus memory, as shown in the diagram above. Basically the same memory appears 4 times in the memory map, with varying degrees of caching, so if the bus address is Cxxxxxxx hex, the memory is uncached. This gives rise to the method:

Allocate memory using mmap with phys addr 0, get virt addr
Convert the virt addr to phys & bus addr
De-allocate the memory
Allocate memory using mmap with same phys addr, in uncached area

I did quite a bit of experimentation with this method, and wasn’t convinced it always works; it was still necessary to include arbitrary delays in the code, otherwise there was still a tendency to sometimes crash.

Eventually my searches for a completely reliable method of getting uncached memory lead me to the VideoCore Mailbox.

VideoCore graphics processor

It may seem strange that I’m tinkering with the graphics processor in order to get uncached memory, but the VideoCore IV Graphics Processing Unit (GPU) controls some primary functionality of the RPi, including the split between main & video memories.

Communication with the GPU is via a confusingly-named ‘mailbox’; this is nothing to do with emails, it is just an ioctl calling mechanism, e.g.

// Open mailbox interface, return file descriptor
int open_mbox(void)
{
   int fd;

   if ((fd = open("/dev/vcio", 0)) < 0)
       FAIL("Error: can't open VC mailbox\n");
   return(fd);
}
// Send message to mailbox, return first response int, 0 if error
uint32_t msg_mbox(int fd, VC_MSG *msgp)
{
    uint32_t ret=0, i;

    for (i=msgp->dlen/4; i<=msgp->blen/4; i+=4)
        msgp->uints[i++] = 0;
    msgp->len = (msgp->blen + 6) * 4;
    msgp->req = 0;
    if (ioctl(fd, _IOWR(100, 0, void *), msgp) < 0)
        printf("VC IOCTL failed\n");
    else if ((msgp->req&0x80000000) == 0)
        printf("VC IOCTL error\n");
    else if (msgp->req == 0x80000001)
        printf("VC IOCTL partial error\n");
    else
        ret = msgp->uints[0];
    return(ret);
}
// Allocate memory on PAGE_SIZE boundary, return handle
uint32_t alloc_vc_mem(int fd, uint32_t size, VC_ALLOC_FLAGS flags)
{
    VC_MSG msg={.tag=0x3000c, .blen=12, .dlen=12,
        .uints={PAGE_ROUNDUP(size), PAGE_SIZE, flags}};
    return(msg_mbox(fd, &msg));
}
// Lock allocated memory, return bus address
void *lock_vc_mem(int fd, int h)
{
    VC_MSG msg={.tag=0x3000d, .blen=4, .dlen=4, .uints={h}};
    return(h ? (void *)msg_mbox(fd, &msg) : 0);
}

The ioctl call requires a 108-byte structure with the command plus data; it returns the response in the same structure:

// Mailbox command/response structure
typedef struct {
    uint32_t len,   // Overall length (bytes)
        req,        // Zero for request, 1<<31 for response
        tag,        // Command number
        blen,       // Buffer length (bytes)
        dlen;       // Data length (bytes)
        uint32_t uints[32-5];   // Data (108 bytes maximum)
} VC_MSG __attribute__ ((aligned (16)));

As you can see, the mailbox functions are quite easy to use; for details of other functionality, see the documentation.

So at last we have a reliable source of uncached memory; for simplicity my software just allocates a single block, which is then subdivided into the control blocks and data needed by the DMA controller.

Code optimisation

One final issue needs to be mentioned in this context; if compiler optimisation is enabled (e.g. gcc command line options -O2 or -O3) then some of the memory accesses may be optimised out, leading to confusing results. For example, you may be using DMA to transfer a data value, and are polling the destination in a tight loop to see when the transfer is complete.

int *destp = ...    // Pointer to somewhere in uncached memory
*destp = 0;
while (*desp == 0)  // While DMA data not received..
    sleep(1);       // ..sleep

On the first poll cycle, the code will read the memory, but subsequent read cycles may be optimised out, so the CPU just re-uses the same data value without re-checking memory.

The solution is simple: declare the variable as volatile, e.g.

volatile int *destp = ...

This ensures that the CPU will always access the memory on every read cycle.

DMA controller

The primary configuration mechanism for the DMA controller is a Control Block (CB). This fully defines the required transfer, including source & destination addresses, data lengths, and the like:

// DMA control block (must be 32-byte aligned)
typedef struct {
    uint32_t ti,    // Transfer info
        srce_ad,    // Source address
        dest_ad,    // Destination address
        tfr_len,    // Transfer length
        stride,     // Transfer stride
        next_cb,    // Next control block
        debug,      // Debug register
        unused;
} DMA_CB __attribute__ ((aligned(32)));
#define DMA_CB_DEST_INC (1<<4)
#define DMA_CB_SRC_INC  (1<<8)

The next_cb address means that you can create a chain of CBs; the controller will work through them all until it encounters a next_cb value of zero.

1st example: memory-to-memory transfer

We’ll start with a really simple operation: a memory-to-memory transfer.

// DMA memory-to-memory test
int dma_test_mem_transfer(void)
{
    DMA_CB *cbp = virt_dma_mem;
    char *srce = (char *)(cbp+1);
    char *dest = srce + 0x100;

    strcpy(srce, "memory transfer OK");
    memset(cbp, 0, sizeof(DMA_CB));
    cbp->ti = DMA_CB_SRC_INC | DMA_CB_DEST_INC;
    cbp->srce_ad = BUS_DMA_MEM(srce);
    cbp->dest_ad = BUS_DMA_MEM(dest);
    cbp->tfr_len = strlen(srce) + 1;
    start_dma(cbp);
    usleep(10);
#if DEBUG
    disp_dma();
#endif
    printf("DMA test: %s\n", dest[0] ? dest : "failed");
    return(dest[0] != 0);
}

The variable virt_dma_mem is pointing to an area of uncached memory, which has been used to house a control block, and the source & destination arrays. The DMA controller starts with that control block, and after a brief delay, the destination is checked to see if the data has been transferred.

I originally thought that the DMA transfer would be so fast that no delay is required, but this isn’t true; some delay is necessary, but even a zero delay is sufficient, i.e. usleep(0), so the 10 microseconds I’ve used is more than adequate.

2nd example: memory-to-GPIO transfer

Assuming the above example works, it is time to try writing to a peripheral, namely a GPIO pin, that can be connected to an LED to provide a simple flashing indication.

On most CPUs you’d write 1 or 0 to a GPIO register to turn the LED on or off, but the Broadcom hardware doesn’t work that way; there is on register to turn it on, and another to turn it off. So we just need to flip the register address between DMA transfers, and the LED will flash.

// DMA memory-to-GPIO test: flash LED
void dma_test_led_flash(int pin)
{
    DMA_CB *cbp=virt_dma_mem;
    uint32_t *data = (uint32_t *)(cbp+1), n;

    printf("DMA test: flashing LED on GPIO pin %u\n", pin);
    memset(cbp, 0, sizeof(DMA_CB));
    *data = 1 << pin;
    cbp->tfr_len = 4;
    cbp->srce_ad = BUS_DMA_MEM(data);
    for (n=0; n<16; n++)
    {
        usleep(200000);
        cbp->dest_ad = BUS_GPIO_REG(n&1 ? GPIO_CLR0 : GPIO_SET0);
        start_dma(cbp);
    }
}

As before, the CB and source data are placed in uncached memory, but the transfer destination is either the ‘set’ or ‘clear’ GPIO registers.

After each on/off transition, the DMA stops, and needs to be restarted with the modified control block.

3rd example: timed triggering

The previous 2 examples are useful demonstrations that DMA is working, but have little practical application since they require significant CPU intervention to keep them running. What we really need is a way of triggering the DMA cycles from a timer, so the transfers carry on automatically while the CPU is doing other tasks.

Unlike most microcontrollers, the Broadcom hardware has no real timers, but it does have a Pulse-Width Modulation (PWM) controller, that can be used instead; it can be programmed to request a data update on a regular basis, i.e. issue a DMA request, and once the update data is received, wait for a fixed time before issuing another request.

That gives us a regular stream of DMA requests at specific intervals, but how do we use that to toggle an LED pin? The answer is that we create 4 control blocks in an endless circular loop:

CB0: clear LED
CB1: write data to PWM controller
CB2: set LED
CB3: write data to PWM controller

You need to bear in mind that the DMA controller will continue processing CBs while its request line is asserted. If we didn’t have CB1 & 3, the DMA cycles would be running continuously, and toggling the LED very fast; this isn’t recommended, since it does use up a lot of memory bandwidth, but on the few occasions I’ve done that, the system seemed to cope quite well, and didn’t crash. With the above arrangement, the controller will execute CB0 & 1, then delay, CB2 & 3, another delay, CB 0 & 1, and so on.

// PWM clock frequency and range (FREQ/RANGE = LED flash freq)
#define PWM_FREQ        100000
#define PWM_RANGE       20000

// DMA trigger test: fLash LED using PWM trigger
void dma_test_pwm_trigger(int pin)
{
    DMA_CB *cbs=virt_dma_mem;
    uint32_t n, *pindata=(uint32_t *)(cbs+4), *pwmdata=pindata+1;

    printf("DMA test: PWM trigger, ctrl-C to exit\n");
    memset(cbs, 0, sizeof(DMA_CB)*4);
    // Transfers are triggered by PWM request
    cbs[0].ti = cbs[1].ti = cbs[2].ti = cbs[3].ti = (1 << 6) | (DMA_PWM_DREQ << 16);
    // Control block 0 and 2: clear & set LED pin, 4-byte transfer
    cbs[0].srce_ad = cbs[2].srce_ad = BUS_DMA_MEM(pindata);
    cbs[0].dest_ad = BUS_GPIO_REG(GPIO_CLR0);
    cbs[2].dest_ad = BUS_GPIO_REG(GPIO_SET0);
    cbs[0].tfr_len = cbs[2].tfr_len = 4;
    *pindata = 1 << pin;
    // Control block 1 and 3: update PWM FIFO (to clear DMA request)
    cbs[1].srce_ad = cbs[3].srce_ad = BUS_DMA_MEM(pwmdata);
    cbs[1].dest_ad = cbs[3].dest_ad = BUS_PWM_REG(PWM_FIF1);
    cbs[1].tfr_len = cbs[3].tfr_len = 4;
    *pwmdata = PWM_RANGE / 2;
    // Link control blocks 0 to 3 in endless loop
    for (n=0; n<4; n++)
        cbs[n].next_cb = BUS_DMA_MEM(&cbs[(n+1)%4]);
    // Enable PWM with data threshold 1, and DMA
    init_pwm(PWM_FREQ);
    *VIRT_PWM_REG(PWM_DMAC) = PWM_DMAC_ENAB|1;
    start_pwm();
    start_dma(&cbs[0]);
    // Nothing to do while LED is flashing
    sleep(4);
}

PWM clock setting

Before leaving the code, it is worth mentioning another area of difficulty: setting the clock frequency of the PWM controller. I arbitrarily chose 100 kHz, since that could be divided by 20,000 to flash the LED at 5 Hz.

The recommended way of setting the clock is using the VideoCore mailbox:

void set_vc_clock(int fd, int id, uint32_t freq)
{
    VC_MSG msg1={.tag=0x38001, .blen=8, .dlen=8, .uints={id, 1}};
    VC_MSG msg2={.tag=0x38002, .blen=12, .dlen=12, .uints={id, freq, 0}};
    msg_mbox(fd, &msg1);
    msg_mbox(fd, &msg2);
}

This method works sometimes, but not always; it can take several attempts to change from one frequency to another, and I don’t understand why.

A fall-back option is to write to the (undocumented) timer registers, which is the method I use by default:

#define USE_VC_CLOCK_SET 0

#if USE_VC_CLOCK_SET
    set_vc_clock(mbox_fd, PWM_CLOCK_ID, freq);
#else
    int divi=(CLOCK_KHZ*1000) / freq;
    *VIRT_CLK_REG(CLK_PWM_CTL) = CLK_PASSWD | (1 << 5);
    while (*VIRT_CLK_REG(CLK_PWM_CTL) & (1 << 7)) ;
    *VIRT_CLK_REG(CLK_PWM_DIV) = CLK_PASSWD | (divi << 12);
    *VIRT_CLK_REG(CLK_PWM_CTL) = CLK_PASSWD | 6 | (1 << 4);
    while ((*VIRT_CLK_REG(CLK_PWM_CTL) & (1 << 7)) == 0) ;
#endif
    usleep(100);

The PWM controller seems to be very sensitive to changes in its clock frequency, so before any change, it is essential to disable it, and wait some time before re-enabling. On one occasion, it locked up completely and just wouldn’t work until I re-powered the board, so care is needed when modifying the clocking code – it is certainly an area that merits further investigation.

Running the code

There is a single source file rpi_dma_test.c on Github here.

You’ll need to change the definition at the top depending on the RPi version you are using:

//#define PHYS_REG_BASE  0x20000000  // Pi Zero or 1
#define PHYS_REG_BASE    0x3F000000  // Pi 2 or 3
//#define PHYS_REG_BASE  0xFE000000  // Pi 4

Then the code can be compiled with GCC, and run with ‘sudo’:

gcc -Wall -o rpi_dma_test rpi_dma_test.c
sudo ./rpi_dma_test

You can optionally compile with -O2 or -O3 optimisation.

To view the results you need to connect an LED (with a 330 ohm resistor in series) to ground and LED_PIN, which I’ve set to GPIO pin 21. This is at the far end of the I/O connector, conveniently next to a ground pin.

Raspberry Pi LED connection

The positive leg of the LED goes to the output pin, which is nearest the camera.

The usual warnings apply when running a program with root privileges -there is a security risk, since it has unrestricted access to all system functions.

To see DMA being used for data acquisition, take a look at my next post.

Copyright (c) Jeremy P Bentham 2020. Please credit this blog if you use the information or software in it.

Zerowi bare-metal WiFi driver part 5: IOCTLs

It has been a long haul, but we are now getting close to doing something useful with the WiFi chip; we just need to tackle the issue of IOCTLs.

You may already be familiar with these from configuring a serial link, or network hardware; they provide a programming interface into a vendor-specific driver. Since the BCM/CTW43xxx chips are intelligent (they have their own CPU) the IOCTL calls are handled directly by the firmware we’ve programmed into the chip. So even though we’re in ‘bare metal’ mode, without an operating system, we still need to handle IOCTLs.

The IOCTL calls are listed in the wwd_wlioctl.h in WICED or WiFi Host Driver, or wlioctl_defs.h, and there are over 300 of them; this post will concentrate on the code that sends IOCTL requests and handles the responses, and we’ll check they are working by doing a quick network scan – more interesting things, like transmission & reception, will have to wait for the next part.

Message structure

When using IOCTL calls, you are essentially writing a data packet to the WiFi RAM, waiting for an acknowledgement, then reading back the response. As you’d expect, there is a specific data format for the requests and responses, though it does have some strange features:

#define IOCTL_MAX_DLEN  256

typedef struct {
    uint8_t  seq,       // sdpcm_sw_header
             chan,
             nextlen,
             hdrlen,
             flow,
             credit,
             reserved[2];
    uint32_t cmd;       // CDC header
    uint16_t outlen,
             inlen;
    uint32_t flags,
             status;
    uint8_t data[IOCTL_MAX_DLEN];
} IOCTL_CMD;

typedef struct {
    uint16_t len;
    uint8_t  reserved1,
             flags,
             reserved2[2],
             pad[2];
} IOCTL_GLOM_HDR;

The best feature of the IOCTL data is that it always starts with a 16-bit length word, followed by the bitwise inverse of that length (least-significant byte first). For example, here is the decode of a request to set a variable ‘bus:rxglom’ to a value of 1:

19.290643 * Cmd 53 A500002C Wr WLAN 08000 len 44
19.290669 * Rsp 53 00001000 Flags 10
  Data  44 bytes: 2b 00 d4 ff 00 00 00 0c 00 00 00 00 07 01 00 00 0f 00 00 00 02 00 02 00 00 00 00 00 62 75 73 3a 72 78 67 6c 6f 6d 00 01 00 00 00 00 *
 IOC_W  44 bytes: seq=0 chan=0 nextlen=0 hdrlen=C flow=0 credit=0 cmd=107 outlen=F inlen=0 flags=20002 status=0 set 'bus:rxglom'
19.290769   Ack 2F FF

You can check this is an IOCTL message by adding the first two bytes to the second two: 002B + FFD4 = FFFF. It uses a command 53 to send a 44-byte request (actually 43 bytes, rounded up to nearest 4-byte value) to the RAD function, containing a header of mostly zeros with an IOCTL number of 107 hex (263 decimal) to set a variable, a null-terminated variable name, then the binary value.

It is then necessary to poll the WiFi chip to check when the response is available, and if so, acknowledge it:

19.291055 * Cmd 53 15404004 Rd BAK  180000:A020 len 4
  Data   4 bytes: 40 00 80 00 *
19.291081 * Rsp 53 00001000 Flags 10
19.291179 * Cmd 53 95404004 Wr BAK  180000:A020 len 4
19.291205 * Rsp 53 00001000 Flags 10
  Data   4 bytes: 40 00 00 00 *
19.291259   Ack 28 3F

The value of 40 hex in backplane register 2020 (A020 for a 32-bit value) shows there is a response, which is acknowledged by writing 40 hex to that register, then the response is read:

19.291377 * Cmd 53 21000040 Rd WLAN 08000 len 64
19.291403 * Rsp 53 00001000 Flags 10
  Data  64 bytes: 2b 00 d4 ff 02 00 00 0c 00 11 00 00 07 01 00 00 0f 00 00 00 00 00 02 00 00 00 00 00 62 75 73 3a 72 78 67 6c 6f 6d 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 *
 IOC_R  64 bytes: seq=2 chan=0 nextlen=0 hdrlen=C flow=0 credit=11 cmd=107 outlen=F inlen=0 flags=20000 status=0 set 'bus:rxglom'

The response is the same length as the request, and when writing to a variable it is largely a copy of the command. When receiving the response, it is important to check that it matches the request, as the two can easily get out of step. Unfortunately the sequence number can’t be used for this purpose (in this example the response is 2, and request is 0) instead the most-significant 16 bits of the ‘flags’ are a ‘request ID’ that should be the same for request & response, while the lower 16 bits are set to 2 for a write cycle, 0 for a read.

Equally strange is that the command length always seems to be the same as the response, so for a short command with a long response (such as ‘ver’) the command is 296 bytes long, just to carry a 3-character name. This is a bit crazy; sometime I’ll experiment with the header fields to see if there is a way round it.

Glom

I’ll admit this word wasn’t in my vocabulary until I encountered it in the WiFi drivers, and I’m still not entirely clear what it means. The transaction above sets ‘rxglom’ to 1, which enables ‘glom’ mode for incoming commands (‘rx’ refers to the WiFi chip command reception, not the host).

After this is set, another header is introduced into commands sent to the WiFi chip; I have accommodated this using another structure, and a union to cover both.

typedef struct {
    uint16_t len;
    uint8_t  reserved1,
             flags,
             reserved2[2],
             pad[2];
} IOCTL_GLOM_HDR;

typedef struct {
    IOCTL_GLOM_HDR glom_hdr;
    IOCTL_CMD  cmd;
} IOCTL_GLOM_CMD;

typedef struct
{
    uint16_t len,           // sdpcm_header.frametag
             notlen;
    union 
    {
        IOCTL_CMD cmd;
        IOCTL_GLOM_CMD glom_cmd;
    };
} IOCTL_MSG;

The good news is that the first 4 bytes of any message remain the same (16-bit length, and its bitwise inverse); the bad news is that the new header is shoehorned in after that, pushing the other headers out by 8 bytes.

Here is an example: getting ‘cur_ethaddr’ which is the 6-byte MAC address:

19.291837 * Cmd 53 A5000038 Wr WLAN 08000 len 56
19.291863 * Rsp 53 00001000 Flags 10
  Data  56 bytes: 38 00 c7 ff 34 00 00 01 00 00 00 00 01 00 00 14 00 00 00 00 06 01 00 00 14 00 00 00 00 00 03 00 00 00 00 00 63 75 72 5f 65 74 68 65 72 61 64 64 72 00 00 00 00 00 00 00 *
 IOC_W  56 bytes: seq=1 chan=0 nextlen=0 hdrlen=14 flow=0 credit=0 cmd=106 outlen=14 inlen=0 flags=30000 status=0 get 'cur_etheraddr'
19.291973   Ack 2F FF

I’m sure there must be some point to the extended header, but right now I’m not at all sure what it is. There doesn’t seem to be any official marker in the glom header to show that it has been included, which makes life difficult for any software attempting to decode the IOCTLs. For the time being, I’m hedging my bets by using a global variable to enable or disable this option, and leaving it disabled; hopefully its true purpose will be clear soon.

Partial data read

If the IOCTL command has a long response, and the software doesn’t read it all, the remainder will still be available for the next read. This can be demonstrated by the version (‘ver’) command; even though it is sent as a single 296-byte block, the Linux driver receives it as one block of 64 bytes, then another of 224:

19.295186 * Cmd 53 A5000128 Wr WLAN 08000 len 296
19.295212 * Rsp 53 00001000 Flags 10
  Data 296 bytes: 28 01 d7 fe 24 01 00 01 00 00 00 00 03 00 00 14 00 00 00 00 06 01 00 00 04 01 00 00 00 00 05 00 00 00 00 00 76 65 72 00 76 65 72 00 00 ..and so on..
 IOC_W 296 bytes: seq=3 chan=0 nextlen=0 hdrlen=14 flow=0 credit=0 cmd=106 outlen=104 inlen=0 flags=50000 status=0 get 'ver'
19.295583   Ack 2F FF
19.295980 * Cmd 52 00000A00 Rd BUS  00005
19.296006 * Rsp 52 00001002 Flags 10 data 02
19.296178 * Cmd 53 15404004 Rd BAK  180000:A020 len 4
  Data   4 bytes: 40 00 80 00 *
19.296204 * Rsp 53 00001000 Flags 10
19.296321 * Cmd 53 95404004 Wr BAK  180000:A020 len 4
19.296347 * Rsp 53 00001000 Flags 10
  Data   4 bytes: 40 00 00 00 *
19.296404   Ack 28 3F
19.296563 * Cmd 53 21000040 Rd WLAN 08000 len 64
19.296589 * Rsp 53 00001000 Flags 10
  Data  64 bytes: 20 01 df fe 05 00 00 0c 00 14 00 00 06 01 00 00 04 01 00 00 00 00 05 00 00 00 00 00 77 6c 30 3a 20 4f 63 74 20 32 33 20 32 30 31 37 20 30 33 3a 35 35 3a 35 33 20 76 65 72 73 69 6f 6e 20 37 2e *
 IOC_R  64 bytes: seq=5 chan=0 nextlen=0 hdrlen=C flow=0 credit=14 cmd=106 outlen=104 inlen=0 flags=50000 status=0 get 'wl0: Oct 23 2017 03:55:53 version 7.'
19.296841 * Cmd 53 210000E0 Rd WLAN 08000 len 224
19.296867 * Rsp 53 00001000 Flags 10
  Data 224 bytes: 34 35 2e 39 38 2e 33 38 20 28 72 36 37 34 34 34 32 20 43 59 29 20 46 57 49 44 20 30 31 2d 65 35 38 64 32 31 39 66 0a 00 00 ..and so on..

This serves to emphasise the important of reading all the data from every response, and checking that the Request ID matches that of the response; it’d be all to easy for the network driver to lose track.

Events

So far, we’ve dealt had a strict one-to-one matching between request and response, but how does the WiFi chip indicate when it has extra data available? For example, a single network scan may generate 10 or 20 data blocks (one for every access point), how does the host know when this data is available? There is mention of an interrupt pin (which we’ll save for a future blog) but how can the driver software check for data pending?

I puzzled over this for some time, on the assumption there must be a special register to indicate this, but in the end it seems that the driver just issues a normal data read; if data is available it can be recognised by the length header, if not zeros are returned.

The WiFi chip has a finite amount of buffer space to queue up such events; this is the ‘credit’ value in the IOCTL header; presumably the network driver should check this to see if events have been lost due to running out of buffers.

Network scan

Finally, we get to do something vaguely useful; scan for WiFi networks. There are 2 types: ‘iscan’ and ‘escan’. The first is an incremental scan, that seems easier to use, but is marked as ‘deprecated’ in some source code. The second is supposed to be more versatile (i.e. more complicated) but is the preferred option, so that is what we’ll be using.

We need to fill in a structure with the scan parameters; due to the large number of networks in the vicinity, I usually scan a single channel:

// WiFi channel number to scan (0 for all channels)
#define SCAN_CHAN       1

typedef struct {
    uint32_t version;
    uint16_t action,
             sync_id;
    uint32_t ssidlen;
    uint8_t  ssid[32],
             bssid[6],
             bss_type,
             scan_type;
    uint32_t nprobes,
             active_time,
             passive_time,
             home_time;
    uint16_t nchans,
             nssids;
    uint8_t  chans[14][2],
             ssids[1][32];
} SCAN_PARAMS;

SCAN_PARAMS scan_params = {
    .version=1, .action=1, .sync_id=0x1234, .ssidlen=0, .ssid={0}, 
    .bssid={0xff,0xff,0xff,0xff,0xff,0xff}, .bss_type=2, .scan_type=1, 
    .nprobes=~0, .active_time=~0, .passive_time=~0, .home_time=~0, 
#if SCAN_CHAN == 0
    .nchans=14, .nssids=0, 
    .chans={{1,0x2b},{2,0x2b},{3,0x2b},{4,0x2b},{5,0x2b},{6,0x2b},{7,0x2b},
      {8,0x2b},{9,0x2b},{10,0x2b},{11,0x2b},{12,0x2b},{13,0x2b},{14,0x2b}},
#else
    .nchans=1, .nssids=0, .chans={{SCAN_CHAN,0x2b}}, .ssids={{0}}
#endif
};

The scan is triggered by sending this data in an ‘escan’ IOCTL call, but first we must tell the chip that we’re interested in the response events. This is done by sending a very large bitfield, with a bit set for each event you want to receive; there are over 140 possible events, so you need to pick the right one. I got the list from whd_events_int.h which is part of the Cypress WiFi Host Driver project; if you don’t know what that is, please refer to part 1 of this blog, which describes all the resources I’m using.

So the code to trigger the scan becomes:

#define EVENT_ESCAN_RESULT  69
#define EVENT_MAX           160
#define SET_EVENT(e)        event_msgs[e/8] = 1 << (e & 7)
uint8_t event_msgs[EVENT_MAX / 8];

SET_EVENT(EVENT_ESCAN_RESULT);
ioctl_set_data("event_msgs", event_msgs, sizeof(event_msgs));
ioctl_set_data("escan", &scan_params, sizeof(scan_params));

Surprisingly easy, until we get back the results of the scan, which has one varying-length record for every WiFi access point found. There is a lot of data, around 300 to 600 bytes per record, so we need to do some heavyweight decoding.

Decoding the scan data

So far, I’ve avoided including any of the standard Cypress / Broadcom header files in my project. This is because any one header file often depends on another 2, which then depends on another 5, and so on… Quite rapidly, you’re including a large chunk of the Operating System which isn’t at all necessary; it just makes the decoding process much harder to follow.

Fortunately for this project, there is a way to avoid these major OS dependencies; use header files that were created for use in embedded systems, namely the Cypress WiFi Host Driver described in part 1 of this blog. Here are the structures that are needed for decoding the scan response data, and the files they’re in:

whd_types.h:
	whd_security, whd_scan_type, whd_bss_type, whd_802_11_band, whd_mac, whd_ssid, whd_bss_type, 
	whd_event_header [-> whd_event_msg], wl_bss_info
whd_events.h:
	whd_event_ether_header, whd_event_eth_hdr, whd_event_msg, whd_event
whd_wlioctl.h:
	wl_escan_result

Additional dependencies for these files are in:

cy_result.h, cyhal_hw_types.h, whd.h

So only 6 extra files need to be included at this stage, and we’ve avoided the unnecessary complexity of an Operating System interface – after all, this driver is supposed to be bare-metal code.

The code to print the MAC address, channel number and SSID (network name) is:

// Escan result event (excluding 12-byte IOCTL header)
typedef struct {
    uint8_t pad[10];
    whd_event_t event;
    wl_escan_result_t escan;
} escan_result;

escan_result *erp = (escan_result *)eventbuff;

n = ioctl_get_event(eventbuff, sizeof(eventbuff));
if (n > sizeof(escan_result))
{
    printf("%u bytes\n", n);
    disp_mac_addr((uint8_t *)&erp->event.whd_event.addr);
    printf(" %2u ", SWAP16(erp->escan.bss_info->chanspec));
    disp_ssid(&erp->escan.bss_info->SSID_len);
}

The scan result data fields are in network-standard byte-order (big endian) so the channel number needs to be byte-swapped.

Running the code

If you want to try out the code so far, you’ll need a Pi ZeroW with a USB-serial cable attached, the arm-none-eabi-gcc compiler and gdb debugger. You can find full details and a simple test program here; it is worth running this before attempting the Zerowi project.

The source code is at https://github.com/jbentham/zerowi, ‘make_scan.bat’ will create zerowi.elf on windows, which is downloaded into the target using the ‘run’ batch file. This executes alpha_speedup.py to accelerate the serial link from 115200 to 921600 baud, then runs Arm gdb using the setup commands in run.gdb.

I have provided Linux scripts ‘make_scan’ and ‘run’, these need to be made executable using ‘chmod +x’. The Alpha debugger does require arm-none-eabi-gdb, which isn’t included in many Linux distributions (including Raspbian Buster) so may need to be built from source.

My Windows system uses serial port COM7, and Linux uses /dev/ttyUSB0; yours may well be different, so you’ll need to change scripts accordingly. If the Cypress firmware is included in the build image (i.e. ‘INCLUDE_FIRMWARE’ is non-zero) then it will take around 10 seconds to load the executable image onto the ZeroW. When the code runs you should see a list of access points; to keep the number of entries low, I only scan a single channel, by default channel 1:

360 bytes
7A:30:D9:96:DA:xx  1 BTWifi-X
460 bytes
84:A4:23:04:81:xx  1 PLUSNET
360 bytes
BC:30:D9:96:DA:xx  1 BTHub6
456 bytes
20:E5:2A:0E:A1:xx  1 Virginia Drive
312 bytes
7A:30:D9:96:DA:xx  1 BTWifi-with-FON
312 bytes
00:1D:AA:C1:75:xx  1 testnet

The last of the these is a special test network I’ll be using in subsequent parts of this blog.

To select another channel, change SCAN_CHAN at the top of zerowi.c; if set to zero, all channels will be scanned.

[Overview] [Previous part] [Next part]

Copyright (c) Jeremy P Bentham 2020. Please credit this blog if you use the information or software in it.

Zerowi bare-metal WiFi driver part 4: loading firmware

In the previous post we sent some commands to the WiFi chip, and got a response. To make the chip do anything useful, we need to program its internal CPU, as it doesn’t have code in ROM.

It does have configuration tables in ROM, that indicate what resources it possesses, and their locations, so the chip variants can all be programmed by a single driver. However, parsing these tables isn’t easy; the simplest code I’ve found is in the the Plan9 driver (see part 1 of this blog for details), and that is moderately impenetrable; here is the parser output for the ZeroW (values in hex):

chip ID A9A6 hex, 43430 decimal
coreid 800, corerev 31
  chipcommon 18000000
coreid 812, corerev 27
  d11ctl 18101000
coreid 829, corerev 15
  sdregs 18002000
  sdiorev 15
coreid 82a, corerev 9
  armcore 82a (ARMcm3)
  armregs 18003000
coreid 80e, corerev 16
  socramregs 18004000

I think these are the Intellectual Property (IP) cores within the chip, and the locations they occupy in the memory map, but in the absence of documentation, a lot of guesswork is required. So I decided to ignore the configuration tables, and just use the same addresses as the Linux driver, after it has done the decode. This makes my driver a lot less flexible, as the addresses will have to be changed for each new chip, but there aren’t many of them, so only a handful of definitions will need changing.

The most important number is the chip ID; it should be A9A6 hex, so it’d be a good idea to check our chip matches that. In part 3 my code did some preliminary SDIO initialisation, now to follow on from that:

// SD function numbers
#define SD_FUNC_BUS     0
#define SD_FUNC_BAK     1
#define SD_FUNC_RAD     2

// Maximum block sizes
#define SD_BAK_BLK_BYTES    64
#define SD_RAD_BLK_BYTES    512

// [0.243831] Set bus interface
sdio_cmd52_writes(SD_FUNC_BUS, BUS_SPEED_CTRL_REG, 0x03, 1);
sdio_cmd52_writes(SD_FUNC_BUS, BUS_BI_CTRL_REG, 0x42, 1);
// [17.999101] Set block sizes
sdio_cmd52_writes(SD_FUNC_BUS, BUS_BAK_BLKSIZE_REG, SD_BAK_BLK_BYTES, 2);
sdio_cmd52_writes(SD_FUNC_BUS, BUS_RAD_BLKSIZE_REG, SD_RAD_BLK_BYTES, 2);

The SD function numbers allow Command 52 & 53 to access 3 different interfaces within the chip: think ‘hardware functions’ rather than ‘software functions’. The SDIO bus interface is configured using the ‘bus’ function, and is set into high-speed mode (as discussed in part 2). Then the block sizes for the backplane (‘BAK’) and radio (‘RAD’) functions are set; these are limited to 64 & 512 bytes by the hardware. These will be used by command 53 when operating in multi-block mode.

#define BAK_BASE_ADDR           0x18000000              // CHIPCOMMON_BASE_ADDRESS

// [17.999944] Enable I/O 
sdio_cmd52_writes(SD_FUNC_BUS, BUS_IOEN_REG, 1<<SD_FUNC_BAK, 1);
if (!sdio_cmd52_reads_check(SD_FUNC_BUS, BUS_IORDY_REG, 0xff, 2, 1))
    log_error(0, 0);
// [18.001750] Set backplane window
sdio_bak_window(BAK_BASE_ADDR);
// [18.001905] Read chip ID 
sdio_cmd53_read(SD_FUNC_BAK, SB_32BIT_WIN, u32d.bytes, 4);

We now use the ‘bus’ function to enable the ‘backplane’ interface; by default, the IP cores in the chip are switched off to conserve power, and they need to be enabled; the second line of code checks that the core has actually powered up (I/O enabled -> I/O ready). Once the backplane function is enabled, we set a window pointing to the common base address (‘chipcommon’ in the Plan9 driver) then do a read, and we get hex values A6 A9 41 15, which is correct. However, some explanation is needed with regard to the backplane window.

Backplane window

You may recall that the commands we’re using here, CMD52 and CMD53, only have a 17-bit address range, yet the chip uses 32-bit addresses internally. The way this is handled is by writing a 24-bit value to 3 of the backplane registers, to act as an offset within the internal space.

// Backplane window
#define SB_32BIT_WIN    0x8000
#define SB_ADDR_MASK    0x7fff
#define SB_WIN_MASK     (~SB_ADDR_MASK)

// Set backplane window, don't set if already OK
void sdio_bak_window(uint32_t addr)
{
    static uint32_t lastaddr=0;
    
    addr &= SB_WIN_MASK;
    if (addr != lastaddr)
        sdio_cmd52_writes(SD_FUNC_BAK, BAK_WIN_ADDR_REG, addr>>8, 3);
    lastaddr = addr;
}
// Do 1 - 4 CMD52 writes to successive addresses
int sdio_cmd52_writes(int func, int addr, uint32_t data, int nbytes)
{
    int n=0;

    while (nbytes--)
    {
        n += sdio_cmd52(func, addr++, (uint8_t)data, SD_WR, 0, 0);
        data >>= 8;
    }
    return(n);
}

It is important to realise that this is a simple windowing scheme where the bottom 15 bits are provided by the offset, and the top 17 bits by the window: the two values aren’t added together. An additional complication (yes, really) is that there are 2 copies of the lower 15-byte address space; 0 – 7fff hex is for byte accesses, and 8000 – ffff hex is for 32-bit word accesses (offset SB_32BIT_WIN).

To give a concrete example, here is the analysis of the RPi driver fetching the CPU ID:

18.001455 * Cmd 52 92001400 Wr BAK  1000A 00
18.001481 * Rsp 52 00001000 Flags 10 data 00
18.001618 * Cmd 52 92001600 Wr BAK  1000B 00
18.001644 * Rsp 52 00001000 Flags 10 data 00
18.001750 * Cmd 52 92001818 Wr BAK  1000C 18 Bak Win 180000
18.001777 * Rsp 52 00001018 Flags 10 data 18
18.001905 * Cmd 53 15000004 Rd BAK  180000:8000 len 4
  Data   4 bytes: a6 a9 41 15 *

You can see the 3 CMD52 write cycles to set the window address, then the 4-byte read cycle, with the offset into the 32-bit area. The ‘win 180000’ and ‘180000:8000’ labels are my analysis code trying to be helpful, by saving the window value, and repeating it at the subsequent read cycle.

Firmware file

There are various firmware versions that could be used (see Cypress WICED) but I’m using the same version as the RPi driver, available here. It is around 300K bytes; eventually, it’ll be stored in the SD card filesystem, but for the time being I wanted a simpler storage mechanism, so attached an external SPI memory device, that can be programmed by a standard RPi utility, and is really easy to read back.

This extra hardware isn’t compulsory; there is an INCLUDE_FIRMWARE option in the source code to link the firmware file into the binary image; the functionality is the same, it just takes longer to load over the target serial link.

The device I used is an EN25Q80B, which has a megabyte of serial flash memory. MikroElektronika sell a small flash click board that is simple to connect to the ZeroW, as follows:

MicroE pi   RPi pin
Gnd          25
3V3          17
SDI          19
SDO          21
SCK          23
CS           24

This can be programmed using the following utilities that are included in the standard Linux distribution:

objcopy -F binary brcmfmac43430-sdio.bin flash.bin --pad-to 0x100000
sudo apt install flashrom
sudo modprobe spi_bcm2835
flashrom -p linux_spi:dev=/dev/spidev0.0,spispeed=1000 -w flash.bin

The version of flashrom I used does issue a warning that the Eon chip isn’t fully supported, but still programs it OK. Reading the chip is really easy:

#define SPI0_BASE       (REG_BASE + 0x204000)
#define SPI0_CS         (uint32_t *)SPI0_BASE
#define SPI0_FIFO       (uint32_t *)(SPI0_BASE + 0x04)
#define SPI0_CLK        (uint32_t *)(SPI0_BASE + 0x08)
#define SPI0_DLEN       (uint32_t *)(SPI0_BASE + 0x0c)
#define SPI0_DC         (uint32_t *)(SPI0_BASE + 0x14)

#define SPI0_CE0_PIN    8
#define SPI0_MISO_PIN   9
#define SPI0_MOSI_PIN   10
#define SPI0_SCLK_PIN   11

// Initialise flash interface (SPI0)
void flash_init(int khz)
{
    gpio_set(SPI0_CE0_PIN, GPIO_ALT0, GPIO_NOPULL);
    gpio_set(SPI0_MISO_PIN, GPIO_ALT0, GPIO_PULLUP);
    gpio_set(SPI0_MOSI_PIN, GPIO_ALT0, GPIO_NOPULL);
    gpio_set(SPI0_SCLK_PIN, GPIO_ALT0, GPIO_NOPULL);
    *SPI0_CS = 0x30;
    *SPI0_CLK = CLOCK_KHZ / khz;
}

// Set / clear SPI chip select
void spi0_cs(int set)
{
    *SPI0_CS = set ? *SPI0_CS | 0x80 : *SPI0_CS & ~0x80;
}

// Start a flash read cycle (EN25Q80 device)
void flash_open_read(int addr)
{
    uint8_t rxdata[4], txdata[4]={3, (uint8_t)(addr>>16), (uint8_t)(addr>>8), (uint8_t)(addr)};
    
    spi0_cs(1);
    spi0_xfer(txdata, rxdata, 4);
}
// Read next block
void flash_read(uint8_t *dp, int len)
{
    while (len--)
    {
        *SPI0_FIFO = 0;
        while((*SPI0_CS & (1<<17)) == 0) ;
        *dp++ = *SPI0_FIFO;
    }
}
// End a flash cycle
void flash_close(void)
{
    spi0_cs(0);
}

If you don’t want to bother with this, just set the INCLUDE_FIRMWARE option in the source code, which links the firmware file into the main executable.

File upload

Before we can upload the code, there is a lot more initialisation to be done; another 34 commands that I won’t be describing here, mainly because I’m having difficulty understanding them in the absence of documentation; for now, the source code is the only explanation you’ll get.

The process of transferring the file is made a bit more complicated by the windowing scheme I described earlier; we have to move that along after every 32K. Command 53 is used in multi-block mode, so one command is issued for multiple data blocks.

// Upload blocks of firmware from flash to chip RAM
int write_firmware(void)
{
    int len, n=0, nbytes=0, nblocks;
    uint32_t addr;

    flash_open_read(0);
    while (nbytes < FIRMWARE_LEN)
    {
        addr = sdio_bak_addr(nbytes);
        len = MIN(sizeof(txbuffer), FIRMWARE_LEN-nbytes);
        nblocks = len / SD_BAK_BLK_BYTES;
		if (nblocks > 0)
        {
            flash_read(txbuffer, nblocks*SD_BAK_BLK_BYTES);
            n = sdio_write_blocks(SD_FUNC_BAK, SB_32BIT_WIN+addr, txbuffer, nblocks);
            if (!n)
                break;
            nbytes += nblocks * SD_BAK_BLK_BYTES;
        }
        else
        {
            flash_read(txbuffer, len);
            txbuffer[len++] = 1;
            sdio_cmd53_write(SD_FUNC_BAK, SB_32BIT_WIN+addr, txbuffer, len);
            nbytes += len;
        }
    }
    flash_close();
    return(nbytes);
}
// Write multiple 64-byte command 53 blocks (max 32K in total)
int sdio_write_blocks(int func, int addr, uint8_t *dp, int nblocks)
{
    int n=0;
    SDIO_MSG rspx, cmd={.cmd53 = {.start=0, .cmd=1, .num=53,
        .wr=1, .func=func, .blk=1, .inc=1, .addrh=(uint8_t)(addr>>15)&3,
        .addrm=(uint8_t)(addr>>7), .addrl=(uint8_t)(addr&0x7f),
        .lenh=(uint8_t)(nblocks>>8)&1, .lenl=(uint8_t)nblocks, .crc=0, .stop=1}};

    clk_0(1);
    add_crc7(cmd.data);
    log_msg(&cmd);
    sdio_cmd_write(cmd.data, MSG_BITS);
    if (sdio_rsp_read(rspx.data, MSG_BITS, SD_CMD_PIN))
    {
        gpio_write(SD_D0_PIN, 4, 0xf);
        gpio_mode(SD_D0_PIN, GPIO_OUT);
        gpio_mode(SD_D1_PIN, GPIO_OUT);
        gpio_mode(SD_D2_PIN, GPIO_OUT);
        gpio_mode(SD_D3_PIN, GPIO_OUT);
        while (n++ < nblocks)
        {
            sdio_block_out(dp, SD_BAK_BLK_BYTES);
            sdio_rsp_read(rspx.data, BLOCK_ACK_BITS, SD_D0_PIN);
            dp += SD_BAK_BLK_BYTES;
            clk_0(2);
        }
        gpio_mode(SD_D0_PIN, GPIO_IN);
        gpio_mode(SD_D1_PIN, GPIO_IN);
        gpio_mode(SD_D2_PIN, GPIO_IN);
        gpio_mode(SD_D3_PIN, GPIO_IN);
    }
    clk_0(1);
    return(n);
}

Once that is complete, we must load in the configuration data, which is available here. A small amount of pre-processing is required, namely removing the comment lines, and replacing all the newline characters with nulls. Since the file is small, command 53 is used in single-block mode.

// Upload blocks of config data to chip NVRAM
int write_nvram(void)
{
    int nbytes=0, len;

    sdio_bak_window(0x078000);
    while (nbytes < config_len)
    {
        len = MIN(config_len-nbytes, SD_BAK_BLK_BYTES);
        sdio_cmd53_write(SD_FUNC_BAK, 0xfd54+nbytes, &config_data[nbytes], len);
        nbytes += len;
    }
    return(nbytes);
}

After another 12 initialisation commands, we can check if the code was loaded OK:

usdelay(50000);
if (!sdio_cmd52_reads(SD_FUNC_BAK, BAK_CHIP_CLOCK_CSR_REG, &u32d.uint32, 1) || u32d.uint8!=0xd0)
    log_error(0, 0);
// [19.190728]
sdio_cmd52_writes(SD_FUNC_BAK, BAK_CHIP_CLOCK_CSR_REG, 0xd2, 1);
sdio_bak_write32(SB_TO_SB_MBOX_DATA_REG, 0x40000);
sdio_cmd52_writes(SD_FUNC_BUS, BUS_IOEN_REG, (1<<SD_FUNC_BAK) | (1<<SD_FUNC_RAD), 1);
sdio_cmd52_reads(SD_FUNC_BUS, BUS_IORDY_REG, &u32d.uint32, 1);
usdelay(100000);
if (!sdio_cmd52_reads(SD_FUNC_BUS, BUS_IORDY_REG, &u32d.uint32, 1) || u32d.uint8!=0x06)
    log_error(0, 0);

If the first value is D0 hex, and the second is 6, then all is well, and after another 21 initialisation commands, we can think about doing something useful with the chip…

[Overview] [Previous part] [Next part]

Copyright (c) Jeremy P Bentham 2020. Please credit this blog if you use the information or software in it.

Zerowi bare-metal WiFi driver part 3: initialisation

This diagram shows the internals of the CYW43xxx chip in simplified form; the important point is that the chip has its own CPU, RAM and ROM; it is a computer-within-a-computer.

In part 2 I mentioned the lack of documentation, and now this becomes a major issue; how to program this complex chip, with no data on its internals. Cypress have partially solved this problem by issuing a standard binary ‘blob’ (roughly 300K bytes) that contains all the code for the embedded CPU; we’ll just be feeding data and control messages into that program, not knowing (or caring) what it is doing to the chip hardware.

I say the problem is ‘partially’ solved because we have to set up the chip to receive this program, upload the code into its RAM, then configure the chip to run it.. a sizeable task, as I’ve discovered.

First steps

The first task is to get the WiFi chip to respond to our commands. The SD and SDIO specifications offer plenty of flowcharts that describe how such a device might be initialised, but I had minimal success with these; they may be applicable to older chips, but maybe the later incarnations of the CYW43xxx just treat the SDIO bus as a convenient parallel interface, rather than slavishly following a specification that was designed for plug-in SD cards.

Then there are the timing issues; a quick glance at the existing code shows many instances where SDIO commands are artificially delayed; after you change something within the chip, it needs time to react before receiving the next command – and if that is sent too quickly, the chip just ignores it, with no error response.

The best way I could find to tackle these problems is to capture all the SDIO commands, responses & data when the Linux driver is running. The capture method is described in the previous part of this blog, and it results in a sizeable file: over 5 gigabytes as a CSV. It contains over 2,200 commands and 13,000 data blocks, so I wrote an application (‘sd_decoder’) to decode the file and display the commands. It turns out that there is a lot of redundancy (for example, the driver loads the 300K binary into the chip, then reads it all back again) so by focusing purely on one WiFi chip, we can make major simplifications.

Fragments of the simplified command sequence can then be replayed into the chip, and eventually, it starts to respond – the first time you get a response from a new chip is a happy day! Here are the first few commands the Linux driver uses to start up the chip:

 0.000331 74 00 00 0c 00 39 * Cmd 52 00000C00 Rd BUS  00006
 0.002759 74 80 00 0c 08 9f * Cmd 52 80000C08 Wr BUS  00006 08
 0.025217 40 00 00 00 00 95 * Cmd  0 00000000
 0.028130 48 00 00 01 aa 87 * Cmd  8 000001AA
 0.028527 45 00 00 00 00 5b * Cmd  5 00000000
 0.028660 3f 20 ff ff 00 ff ? Rsp 63 20FFFF00
 0.028875 45 00 20 00 00 3d * Cmd  5 00200000
 0.029007 3f a0 ff ff 00 ff ? Rsp 63 A0FFFF00
 0.029228 43 00 00 00 00 21 * Cmd  3 00000000
 0.029353 03 00 01 00 00 eb * Rsp  3 00010000
 0.029690 47 00 01 00 00 dd * Cmd  7 00010000
 0.029815 07 00 00 1e 00 a1 * Rsp  7 00001E00

To explain the format; firstly the time in seconds, then 6 command/response bytes, ‘*’ to indicate CRC is correct, or ‘?’ if incorrect, then a partial decode.

A few points to note:

  • The first 4 commands produce no responses.
  • There are jumps in the timestamp, presumably caused by intentional delays.
  • CMD5 is described in the SDIO specification as enabling I/O mode, but the response we get has an incorrect CRC.
  • The CMD3 – CMD7 sequence is described in the SD specification; it is the way that a host selects one card from multiple card slots; CMD3 gets a Relative Card Address (RCA), then command 7 selects the card using that address.

It’d be nice to understand why two of the commands have failed CRCs, but the Linux driver ignores that error, so I will as well. The above sequence is implemented in my code as follows:

int rca;
sdio_cmd52(SD_FUNC_BUS, 0x06, 0, SD_RD, 0, 0);   
usdelay(20000);
sdio_cmd52(SD_FUNC_BUS, 0x06, 8, SD_WR, 0, 0);   
usdelay(20000);
sdio_cmd(0, 0, 0);
sdio_cmd(8, 0x1aa, 0);
// Enable I/O mode
sdio_cmd(5, 0, 0);
sdio_cmd(5, 0x200000, 0);
// Assert SD device
sdio_cmd(3, 0, &resp);
rca = SWAP16(resp.rsp3.rcax);
sdio_cmd7(rca, 0);

Note the time delays; it is tempting to reduce them, but I have found from bitter experience that this can result in major problems much later on (as a critical setting has been ignored), so I wouldn’t recommend doing that.

Raspberry Pi I/O

Now it is necessary to translate the SDIO commands into hardware I/O cycles. The good news about bare-metal programming is that is isn’t necessary to use a fancy driver, or seek permission from the operating system; we can just control the I/O directly.

The primary source of information is the ‘BCM2835 ARM Peripherals’ document; armed with that and knowledge of the I/O base address (0x20000000 for the Pi ZeroW) we can create suitable low-level functions.

// Addresses
#define REG_BASE    0x20000000      // Pi Zero (0x3F000000 for Pi 3)
#define GPIO_BASE       (REG_BASE + 0x200000)
#define GPIO_MODE0      (uint32_t *)GPIO_BASE
#define GPIO_SET0       (uint32_t *)(GPIO_BASE + 0x1c)
#define GPIO_CLR0       (uint32_t *)(GPIO_BASE + 0x28)
#define GPIO_REG(a)     ((uint32_t *)a)

// Mode values
#define GPIO_IN         0
#define GPIO_OUT        1
#define GPIO_ALT0       4
#define GPIO_ALT1       5
#define GPIO_ALT2       6
#define GPIO_ALT3       7

// Configure pin as input or output
void gpio_mode(int pin, int mode)
{
    uint32_t *reg = GPIO_REG(GPIO_MODE0) + pin / 10;
    int shift = (pin % 10) * 3;

    *reg = (*reg & ~(7 << shift)) | (mode << shift);
}

// Set an O/P pin
void gpio_out(int pin, int val)
{
    uint32_t *reg = (val ? GPIO_REG(GPIO_SET0) : GPIO_REG(GPIO_CLR0)) + pin/32;

    *reg = 1 << (pin % 32);
}

// Get an I/P pin value
uint8_t gpio_in(int pin)
{
    uint32_t *reg = GPIO_REG(GPIO_LEV0) + pin/32;

    return (((*reg) >> (pin % 32)) & 1);
}

Configuring a pin as an input or output is done by setting a 3-bit values. Writing 1 or 0 to a pin is (sadly) done using separate set & clear registers; this does make any speed-optimisations (e.g. direct DMA to I/O ports) significantly harder, so I haven’t tried this yet.

Running under Linux

Although the whole purpose of this project is to run without Linux, after I’d written the above code, I did wonder whether it’d speed up development if I ran it under Linux with the WiFi interface shut down, using mmap() to gain access to the devices at low level.

This experiment failed; I never got reliable communications with the WiFi chip, and the operating system had a tendency to crash after my code was run. This isn’t too surprising, since the whole point of the OS is to control the hardware, and having a user-mode program controlling it as well, is really asking for trouble.

Timer

In addition to I/O cycles, we need a microsecond timing reference, that can be used to provide accurate delays. Fortunately there is a 32-bit register clocked at 1 MHz that is ideal for the purpose.

#define USEC_BASE       (REG_BASE + 0x3000)
#define USEC_REG()      ((uint32_t *)(USEC_BASE+4))

// Delay given number of microseconds
void usdelay(int usec)
{
    int ticks;

    ustimeout(&ticks, 0);
    while (!ustimeout(&ticks, usec)) ;
}

// Return non-zero if timeout
int ustimeout(int *tickp, int usec)
{
    int t = *USEC_REG();

    if (usec == 0 || t - *tickp >= usec)
    {
        *tickp = t;
        return (1);
    }
    return (0);
}

SDIO output

The Raspberry Pi CPU is sufficiently fast that we can easily toggle the clock line at 500 kHz, while shifting the command bits out.

#define SD_CLK_DELAY    1   // Clock on/off time in usec

// Write command to SD interface
void sdio_cmd_write(uint8_t *data, int nbits)
{
   uint8_t b, n;

    gpio_mode(SD_CMD_PIN, GPIO_OUT);
    for (n=0; n<nbits; n++)
    {
        if (n%8 == 0)
            b = *data++;
        gpio_out(SD_CMD_PIN, b & 0x80);
        b <<= 1;
        usdelay(SD_CLK_DELAY);
        gpio_out(SD_CLK_PIN, 1);
        usdelay(SD_CLK_DELAY);
        gpio_out(SD_CLK_PIN, 0);
    }
    gpio_mode(SD_CMD_PIN, GPIO_IN);
}

This code could be made much faster, by eliminating the delays. When writing the data output function I took a more aggressive approach to the timing; the transfers are error-free with a 2 MHz clock (8 Mbit/s of data) and could go faster with some optimisation.

Reception is a lot more tricky, handling command responses, data on CMD53 read-cycles, and acknowledgements on write-cycles. This requires multiple state-machines, triggered by the clock edges and ‘start’ bit detection; see the source code for details.

The 7- and 16-bit CRC calculations for commands & data have already been explained in part 2 of this blog.

[Overview] [Previous part] [Next part]

Copyright (c) Jeremy P Bentham 2020. Please credit this blog if you use the information or software in it.

Zerowi bare-metal WiFi driver part 2: SDIO

Hardware

I’m starting this project with the Raspberry Pi ZeroW, which uses the Cypress WiFi chip CYW43438. It interfaces to the ARM processor using Secure Digital I/O (SDIO), which consists of the following signals:

  • Clock (1 line, O/P from CPU)
  • Command (1 line, I/O)
  • Data (4 lines, I/O)

Later in this blog, I’ll be describing what these pins do, in case you are a newcomer to the strange world of SDIO.

The I/O bit numbers are defined in the DeviceTree file for the board:

sdio_pins {
    brcm,pins = <0x00000022 0x00000023 0x00000024 0x00000025 0x00000026 0x00000027>;
    brcm,function = <0x00000007>;
    brcm,pull = <0x00000000 0x00000002 0x00000002 0x00000002 0x00000002 0x00000002>;
    phandle = <0x00000019>;
};

The ‘pull’ settings show that pullup resistors are enabled for pin 23 to 27 hex (GPIO35 to 39), and an initial guess would be that these pins are the command and data, while 22 hex (GPIO34) is the clock.

The datasheet mentions a power-on signal, and a quick trawl on the Web suggests that this could be GPIO41, which must be high to power up the WiFi interface. There is also mention of a low-speed (32 kHz) clock that may be needed when waking up the chip from low-power mode; it turns out this is on GPIO43. This can be verified by dumping the I/O configuration registers when the WiFi interface is running:

  20300000: 0013D660 00017030 A5040030 353A0002 00001000 00000000 00000000 00000000
  20300020: 00000000 01FF0000 00000F02 000E0207 00000000 00FF0133 00FF0133 00000000

Each pin has a 3-bit mode value, that shows whether it being used for simple input, output, or is connected to an internal peripheral (ALT0 – 5). The values above can be decoded by referring to the ‘BCM2835 ARM Peripherals’ data sheet, but an easier way is to use the ‘pigs’ front-end for the PIGPIO library, thus:

sudo pigpiod   [load PIGPIO daemon]
pigs mg 34     [get mode of GPIO pin 34]
7              [returned value 7: pin is ALT3]
pigs mg 43     [get mode of GPIO pin 43]
4              [returned value 4: pin is ALT0]

Pins 34 to 39 are all set to ALT3, which is unhelpfully labelled in the BCM2835 datasheet as ‘reserved’; in reality this means they are connected to the (undocumented) Arasan SD controller. GPIO43 is configured as ALT0, which is the clock source GPCLK2, configured for 32.768 kHz.

Attaching a logic analyser

To understand what the Linux driver is doing, I need to attach a logic analyser to the SDIO bus. This isn’t easy on most boards; the interface runs very fast (up to 50 MHz) so the only means of attachment is by soldering onto extremely small surface-mount components, that can easily be damaged.

However, the Pi Zerow has some interesting pads on the underside.

SDIO pads

Those 7 gold circles are clearly attached to some internal signals, since they have conductive holes (known as ‘vias’) to tracks on other layers. Also, they’re in the right area for the SDIO interface, and it is possible they’re needed for testing the WiFi/ Bluetooth interface after the PCB is assembled. Monitoring these signals with WiFi running proved that they do have almost all of the SDIO signals, aside from the most important one: the clock. Further probing suggested that the only way to pick up that signal is on the other side of the board at a resistor, but connecting to this point is tricky; you need good surface-mount soldering skills to avoid damaging the board.

SDIO clock connection

The main problem with the logic analyser interface is the sheer volume of data that’ll be accumulated. The boot process takes around a minute, with sporadic activity on the SDIO interface; catching all that, with a data rate of 50 MHz, would require a very complicated and/or expensive setup. Fortunately, the Raspberry Pi has an ‘overclocking’ setting in the boot file config.txt, which sets the clock rate to be used when the OS requests 50 MHz. This doesn’t just speed up the interface; a value of 1 or 2 MHz can be used to slow it right down, e.g.

# Add to /boot/config.txt:
dtparam=sdio_overclock=2

This allows a lower-cost analyser to be used (see part 1 of this blog for details) – and surprisingly, the change doesn’t make a lot of difference to the boot-time, since there are long pauses in SDIO activity, where the OS is doing other things. This can be seen by zooming the analyser display out to the maximum, showing 50 seconds of data:

SDIO activity during Linux boot

The bottom trace is the clock, the next is the command (CMD) line, then there are the 4 data lines. Despite the long periods with no activity, there is a lot going on: over 2,200 commands and 13,000 data blocks are being exchanged between the CPU and the WiFi chip.

SDIO protocol

If, like me, you have some experience of the Serial Peripheral Interface (SPI), you may expect SDIO to be similar, in that it uses a clock line to synchronise the sender & receiver; the rising edge of the clock indicates that the data is stable, and can be read by the receiver.

However, there are a few key differences:

  • Bi-directional. All the lines, apart from the clock, are bi-directional; either side can drive them.
  • Command and data lines. There are separate lines for commands and data, and the 4 data lines act as a 4-bit parallel bus.
  • Start & end bits. Instead of the SPI chip-select, the data and command lines idle high, then go low to signal the start of a transfer; this is referred to as a ‘start bit’, and is a single bit-time with a value of zero. At the end of the transfer there is a single bit with a value of 1, an ‘end bit’.
  • Format. The format of SDIO commands and responses is standardised, with specific meaning to the transferred bytes.

It is well worth reading the SDIO specification; at the time of writing, the latest version that is available from the SD Association is “SD Specifications Part E1, SDIO Simplified Specification Version 3.00”. For a few of the commands you need to refer back to the “SD Specifications Part 1 Physical Layer Simplified Specification”, for example:

SD command and response

This is SD command 3 (generally abbreviated to CMD3) and response 6 (R6) from the target (WiFi chip). Both are specified at being 48 bits long, and you can see they begin with a 0 start-bit, and finish with a 1 end-bit. Between the two, the command line is briefly idle. It is a bit confusing that the reply to a command 3 is not a response 3; this is because there are a lot of commands (over 50) but many of them share the same response format, so only 7 possible responses have been defined.

The most common commands used in the SDIO interface are CMD52 and 53. Command 52 is used to read or write a single 8-bit value, while CMD53 transfers blocks of data, either singly or in batches. The following trace shows command 53 reading a single block of 4 data bytes; the command and response look similar to command 52, but there is also activity on the data lines, starting with a 4-bit value of zero, and ending with F hex.

SDIO command 53

SDIO interface code

In the absence of the necessary documentation, writing code for the ‘Arasan’ SD controller on the Raspberry Pi would be quite fraught, so I decided to use direct control (‘bit-bashing’ or ‘bit-banging’) of the I/O lines. The more experienced among you might be thinking this is a really bad idea, as it can be very CPU-intensive and slow, but I believe that the end-result (for example, booting the WiFi chip from scratch in 1 second) vindicates my decision – and if you want to use the controller, you can modify my code to do so.

The bit-patterns within the SDIO commands and responses are quite complex, and the code I’ve seen makes heavy use of bit-masking and shifting to combine the individual values into a single message. I’m not a fan of this approach, and prefer to use C language bitfields. For example, CMD52 has the following fields in the 6-byte message:

Start:             1 bit  (always 0)
Direction:         1 bit  (1 for command, 0 for response)
Command index:     6 bits (52 decimal for command 52)
R/W flag:          1 bit  (0 for read, 1 for write)
Function number:   3 bits (select the bus, backplane or radio interface)
RAW flag:          1 bit  (1 to read back result of write)
Unused:            1 bit
Register address: 17 bits (128K address space)
Unused:            1 bit
Data value:        8 bits (byte to be written, unused if read cycle)
CRC:               7 bits (cyclic redundancy check)
End:               1 bit  (always 1)

You’ll see that the values don’t all line up on convenient 8-bit boundaries; furthermore the data sheet defines the values with most-significant value first, whereas standard C structures have least-significant first.

My solution is to use macros to reverse the order of bits in the byte, so the command structure looks similar to the specification. These are used to create structures for the commands and responses, which are combined in a union.

#define BITF1(typ, a)             typ a
#define BITF2(typ, a, b)          typ b, a
#define BITF3(typ, a, b, c)       typ c, b, a
..and so on..

typedef struct
{
    BITF3(uint8_t,  start:1, cmd:1,   num:6);
    BITF5(uint8_t,  wr:1,    func:3,  raw:1, x1:1, addrh:2);
    BITF1(uint8_t,  addrm);
    BITF2(uint8_t,  addrl:7, x2:1);
    BITF1(uint8_t,  data);
    BITF2(uint8_t,  crc:7,   stop:1);
} SDIO_CMD52_STRUCT;

typedef union 
{
    SDIO_CMD52_STRUCT   cmd52;
    SDIO_RSP52_STRUCT   rsp52;
    SDIO_CMD53_STRUCT   cmd53;
    uint8_t             data[MSG_BYTES+2];
} SDIO_MSG;

The code to split the 17-bit address into 3 bytes is still a bit messy, but the structure definition does simplify the process of creating a command:

// Send SDIO command 52, get response, return 0 if none
int sdio_cmd52(int func, int addr, uint8_t data, int wr, int raw, SDIO_MSG *rsp)
{
    SDIO_MSG cmd={.cmd52 = {.start=0, .cmd=1, .num=52,
        .wr=wr, .func=func, .raw=raw, .x1=0, .addrh=(uint8_t)(addr>>15 & 3),
        .addrm=(uint8_t)(addr>>7 & 0xff), .addrl=(uint8_t)(addr&0x7f), .x2=0,
        .data=data, .crc=0, .stop=1}};

    return(sdio_cmd_rsp(&cmd, rsp));
}

For speed, the CRC is created using a byte-wide lookup table in RAM, which is computed on startup:

#define CRC7_POLY    (uint8_t)(0b10001001 << 1)

uint8_t crc7_table[256];

// Initialise CRC7 calculator
void crc7_init(void)
{
    for (int i=0; i<256; i++)
        crc7_table[i] = crc7_byte(i);
}

// Calculate 7-bit CRC of byte, return as bits 1-7
uint8_t crc7_byte(uint8_t b)
{
    uint16_t n, w=b;

    for (n=0; n<8; n++)
    {
        w <<= 1;
        if (w & 0x100)
            w ^= CRC7_POLY;
    }
    return((uint8_t)w);
}

// Calculate 7-bit CRC of data bytes, with l.s.bit as stop bit
uint8_t crc7_data(uint8_t *data, int n)
{
    uint8_t crc=0;

    while (n--)
        crc = crc7_table[crc ^ *data++];
    return(crc | 1);
}

Data CRC

The data transfer includes a CRC for every line, for example this is the transfer of the 4 bytes A6, A9, 41, and 15 hex.

Command 53 data read

A total of 12 bytes are transferred, because each data line has an added 16-bit CRC. This was a bit of a headache, since splitting the 4-bit data into individual 1-bit values for CRC calculation would considerably slow down the command generation & checking. Fortunately there is a easy way to calculate & check the CRC for each line, while still keeping the 4 values together. This comes from the realisation that the exclusive-or operation in the CRC doesn’t care what order the bits are in; we can rearrange the bits to match our data. So we can compute all 4 CRCs using a single 64-bit value:

Bit 0: bit 0 of 1st CRC
Bit 1: bit 0 of 2nd CRC
Bit 2: bit 0 of 3rd CRC
Bit 3: bit 0 of 4th CRC
Bit 4: bit 1 of 1st CRC
..and so on, up to..
Bit 63: bit 15 of 4th CRC

Once they have been computed, the 4 CRCs are transmitted by just shifting out the next 4 bits of the 64-bit value. To speed up the calculation, a 4-bit lookup table is initialised on startup:

#define CRC16R_POLY  (1<<(15-0) | 1<<(15-5) | 1<<(15-12))

uint64_t qcrc16r_poly, qcrc16r_table[16];

// Initialise bit-reversed CRC16 lookup table for 4-bit values
void qcrc16r_init(void)
{
    qcrc16r_poly = quadval(CRC16R_POLY);
    for (int i=0; i<(1<<SD_DATA_PINS); i++)
        qcrc16r_table[i]  = (i & 8 ? qcrc16r_poly<<3 : 0) |
                            (i & 4 ? qcrc16r_poly<<2 : 0) |
                            (i & 2 ? qcrc16r_poly<<1 : 0) |
                            (i & 1 ? qcrc16r_poly<<0 : 0);
}

// Spread a 16-bit value to occupy 64 bits
uint64_t quadval(uint16_t val)
{
    uint64_t ret=0;

    for (int i=0; i<16; i++)
        ret |= val & (1<<i) ? 1LL<<(i*4) : 0;
    return(ret);
}

Now when transmitting, the 64-bit (i.e. 4 x 16-bit) CRC is updated with every 4-bit value:

uint64_t qcrc=0;

// For each 4-bit value 'd':
qcrc = (qcrc >> 4) ^ qcrc16r_table[(d ^ (uint8_t)qcrc) & 0xf];

After the data has been sent, the CRC values are transmitted:

for (n=0; n<16; n++)
{
    output((uint8_t)qcrc & 0xf);
    qcrc >>= 4;
}

Bulk transfers

Command 53 can be used to transfer a single block (where the block size is specified in the command) or multiple blocks (where the block size has previously been set, and the number of blocks is specified in the command).

If the CPU is sending a single command, then writing multiple blocks to the WiFi chip, how does it check that the blocks are being received and processed OK? The answer is that when writing blocks, the recipient generates a brief acknowledgement back. Here is an example of a CMD53 write.

Command 53 write

It is a bit difficult to see what is going on; the command and response are similar to all the others, but then (after a surprisingly long pause) the data is transferred from the CPU to the WiFi chip. Zooming in on that data:

Command 53 write data

4 bytes with the values 3, 0, 0, 0, are being transferred, then 8 bytes of CRC. However, after that, the recipient acknowledges the received data by driving the least-significant data line with a bit value of 00101000 00111111 (28 3F hex). To be honest, I haven’t been able to find a proper description of these bits; I assume there is a single byte value, then the recipient holds the line low until it has finished processing, but the meaning of the byte bits isn’t at all clear. So for the time being, my code reads in the byte value, then waits for the line to go high, effectively treating it as a ‘busy bit’.

Clock polarity

A small but significant detail is the relationship between the data changes and the clock edges – when is the data stable so it can be read? I previously suggested that the data is read on the positive clock-edge, but look at this trace showing the transition between the command and the response:

Command and response clocking

For the command, the data changes on the negative-going clock edge, so can be read on the positive-going edge. The response appears to be the opposite way around, with the data changing on the positive-going edge – what is going on?

The answer is that the WiFi chip has been set to ‘SDIO High-Speed’ mode; the data changes very shortly after the positive-going edge of the clock, so as to enable fast transfers. The timing is described in the chip data sheet if you want to know the details, but the end result is that the logic analyser isn’t fast enough to capture the gap between clock & data changes, so the software that analyses the logic traces has to use the last state before the clock goes high.

Bit bashing

The bit-bash (or bit-bang) code was quite difficult to write; it has to toggle the clock line, feed out the single-bit command, then get the response whilst simultaneously sending or receiving the 4-bit data. Also, although the examples above show the data being shorter than the response, in reality it can be considerably longer, up to 512 bytes, so will finish long after the activity on the command line. Then there is the issue of the acknowledgements of block data writes, and the need for a timeout in case the chip goes unresponsive…

There is no point describing the code here; if you are interested, take a look at the source. In theory, it’d be a good idea to replace it with a driver for the Arasan SD controller, but I’m not sure there would be a large speed gain – the Linux driver seems to spend a lot of time idle, waiting for the SD controller to complete a task. Also the bit-bashing code is more universal: it shouldn’t be too difficult to port it to other processors such as the STM32, which is frequently paired with the Cypress chip in a standalone module.

SPI interface

For completeness, I need to mention that according to the data sheet, the WiFi chip has a Serial Peripheral Interface (SPI), that can be used instead of SDIO. This is enabled by sending a reset command to the chip, while certain I/O lines are held in specific states.

I originally thought this interface would be easier to use than SDIO, but all my attempts to get it working failed. Also, the SPI connections don’t seem to line up with the SPI master in the BCM2835, so the interface would have to be bit-bashed, which would be really slow as the data bus is only 1-bit-wide. So I abandoned SPI, and focused exclusively on SDIO.

[Overview] [Previous part] [Next part]

Copyright (c) Jeremy P Bentham 2020. Please credit this blog if you use the information or software in it.

Zerowi bare-metal WiFi driver part 1: resources

The WiFi chips on the Raspberry Pi boards are from the Broadcom BCM43xxx range, which has been taken over by Cypress, and renamed CYW43xxx. The Pi ZeroW uses the CYW43438, the PDF data sheet is here.

The obvious starting point for creating a new driver is the existing Linux driver, known as ‘Broadcom Full MAC’, or BRCMFMAC. The source is available here.

A more recent driver is included in the Cypress WICED development system. This is based on Eclipse, and is very comprehensive, covering a wide range of wireless chips and functionality.

A more compact version of the code is available as the Cypress Wifi Host Driver, which is intended for integration in real-time systems.

Another (highly unusual) WiFi driver is available for the Plan9 operating system, contained within a single file ether4330.c. This has a large number of unconventional operating-system dependencies, and would require significant modifications to run on a bare-metal platform, but is interesting as the author has succeeded in creating some remarkably compact code.

It is great to have such a range of source-code available, but in practice it has been of minimal use in this project; even the simplest versions are exceedingly complex, with hidden dependencies & timing issues, so rather than simplifying existing code, I have written all the code from scratch.

Documentation on the Broadcom/Cypress chips is very limited; the data sheet gives minimal information about the chip internals. There is a programming manual, but that only available from Cypress under a confidentiality agreement. I haven’t had access to that document, as it would place severe restrictions on what I can write in this blog. So I have just used freely-available information, logic analysis, and a lot of trial-and-error.

With regard to a logic analyser, the requirements for this project are a bit demanding. The Raspberry Pi ZeroW takes nearly a minute to boot, so a recording of the hardware activity will be quite long, overflowing the memory of most logic analysers. So it is necessary to using a ‘streaming’ analyser, that can send data continuously to a PC’s hard disk; I’ve been using the DSLogic Basic from DreamSourceLab, as it can stream 8-bit values at 20 megasamples per second, and has an easy-to-use GUI based on Sigrok, but a simpler device streaming 10 megasamples per second might be adequate for most analysis tasks.

Sigrok pulseview (the open-source logic analyser GUI) has a built-in Secure Digital (SD) decoder, but this is of limited use on the Secure Digital I/O (SDIO) interface of the Broadcom/Cypress chips, so for analysis I export the data as a very large CSV file (over 5 gigabytes!), then use my own software tools written in the C language to analyse it – Python would be much too slow.

A variant of this analysis code has been used to draw the logic analyser diagrams for this blog – they are all derived from real-world data.

The data analysis tools run on a Windows PC, and have been compiled using gcc 7.4.0 or 8.2.0 from the cygwin project. I suspect the code would also run under Linux with minimal modifications, but haven’t had the time to try this out. The programs can be compiled directly from the command-line, e.g. to produce sd_decoder.exe:

gcc -Wall -o sd_decoder sd_decoder.c

If you want to understand the SDIO interface, the specifications are essential reading; they are available at the SD Association.

My source code is at https://github.com/jbentham/zerowi

[Overview] [Next part]

Copyright (c) Jeremy P Bentham 2020. Please credit this blog if you use the information or software in it.

Raspberry Pi bare-metal programming using Alpha

‘Bare-metal’ is programming without an operating system – running the code directly on the hardware, without the usual device drivers.

I’ve been developing a bare-metal driver for the WiFi chip on the Raspberry Pi ZeroW, and needed a method of downloading & debugging the code. Alpha by Farjump seemed ideal for the purpose; it is a small remote GDB server, that can be controlled by a Windows or Linux PC, using a simple 2-wire serial link.

In this blog I’ll describe how to set up Alpha, and give some tips to maximise the functionality of this excellent application. I’ve been using Windows as a development platform, so this text is biased in that direction, but much of the information is applicable to Linux as well.

Limitations

So far, I have only had success running Alpha on the original Pi version 1, and the ZeroW; for example, it didn’t work on version 3 hardware. This may be due to errors on my part; I’m not sure which board versions are actually supported by the current release.

Hardware connection

You need a 3-wire serial connection (ground, transmit & receive) at 3.3-volt logic levels. Any USB-to-serial adaptor should work, so long as it has a 3.3V output, not 5 volt or RS-232.

I use an FTDI cable for the purpose, the TTL-232R-RPi, which has just black, yellow and red wires connected as follows:

Raspberry Pi Alpha connections

These are labelled from the perspective of the Raspberry Pi, so the Txd line will go to Rxd on your serial adaptor, and vice-versa. Take care when connecting, due to the closeness of the 5 volt power pins; they could cause serious damage.

Installation

Clone or download Alpha from https://github.com/farjump/raspberry-pi

You just need 4 files in the root directory of the SDHC card that is plugged into the Raspberry Pi:

An install script for Linux is provided (scripts/install-rpi-boot.sh) but I had no luck with this, so had to do a manual install. Alpha.bin and config.txt come from the Alpha distribution ‘boot’ directory, bootcode.bin and start.elf are copied from the root directory of a Raspbian distribution.

The Raspberry Pi boot directory is FAT32 formatted, so you don’t have to run Linux; you can plug the SDHC card into a USB adaptor on a Windows PC, and copy the required files across.

When you boot the system, nothing seems to happen; you need to use the serial link to check alpha is working.

Compiler

The compiler I’ve been using is gcc-arm-none-eabi, version 7-2018-q2-update. Installation on Raspbian Buster just requires:

sudo apt install gcc-arm-none-eabi

On Windows, download from here; this places the tools in the directory

C:\Program Files (x86)\GNU Tools Arm Embedded\7 2018-q2-update\bin

Check that this directory in included in your search path by opening a command window, and typing

arm-none-eabi-gcc -v
arm-none-eabi-gdb -v

If not found, close the window, add to the PATH environment variable, and retry.

For more complicated projects, you’ll probably be using Makefiles, and on Windows, will need to install ‘make’ from here. As with GCC, check that it is included in your executable path by opening a new command window, and typing

make -v

Building a project

The SDK files are in the sdk sub-directory of the Alpha distribution; for simplicity, you can just copy it to create an identical sdk sub-directory in your project directory.

We need something to compile, so here is a simple program alpha_test.c to flash the LED on a Pi ZeroW at 1 Hz.

// Simple test of Raspberry Pi bare-metal I/O using Alpha
// From iosoft.blog, copyright (c) Jeremy P Bentham 2020

#include <stdint.h>
#include <stdio.h>

#define REG_BASE    0x20000000      // Pi Zero

#define GPIO_BASE   (REG_BASE + 0x200000)
#define GPIO_MODE0  (uint32_t *)GPIO_BASE
#define GPIO_SET0   (uint32_t *)(GPIO_BASE + 0x1c)
#define GPIO_CLR0   (uint32_t *)(GPIO_BASE + 0x28)
#define GPIO_LEV0   (uint32_t *)(GPIO_BASE + 0x34)

#define GPIO_REG(a) ((uint32_t *)a)

#define USEC_BASE   (REG_BASE + 0x3000)
#define USEC_REG()  ((uint32_t *)(USEC_BASE+4))

#define GPIO_IN     0
#define GPIO_OUT    1

#define LED_PIN     47

void gpio_mode(int pin, int mode);
void gpio_out(int pin, int val);
uint8_t gpio_in(int pin);
int ustimeout(int *tickp, int usec);

int main(int argc, char *argv[])
{
    int ticks=0;
    
    gpio_mode(LED_PIN, GPIO_OUT);
    ustimeout(&ticks, 0);
    printf("\nAlpha test");
    while (1)
    {
        if (ustimeout(&ticks, 500000))
        {
            gpio_out(LED_PIN, !gpio_in(LED_PIN));
            putchar('.');
            fflush(stdout);
        }   
    }
}

// Set input or output
void gpio_mode(int pin, int mode)
{
    uint32_t *reg = GPIO_REG(GPIO_MODE0) + pin / 10, shift = (pin % 10) * 3;

    *reg = (*reg & ~(7 << shift)) | (mode << shift);
}

// Set an O/P pin
void gpio_out(int pin, int val)
{
    uint32_t *reg = (val ? GPIO_REG(GPIO_SET0) : GPIO_REG(GPIO_CLR0)) + pin/32;

    *reg = 1 << (pin % 32);
}

// Get an I/P pin value
uint8_t gpio_in(int pin)
{
    uint32_t *reg = GPIO_REG(GPIO_LEV0) + pin/32;

    return (((*reg) >> (pin % 32)) & 1);
}

// Return non-zero if timeout
int ustimeout(int *tickp, int usec)
{
    int t = *USEC_REG();

    if (usec == 0 || t - *tickp >= usec)
    {
        *tickp = t;
        return (1);
    }
    return (0);
}

// EOF

For details of the built-in peripherals, see the ‘BCM2835 ARM Peripherals’ document, available here.

My code polls a microsecond time register, toggling the LED when it reaches a certain value. This allows the CPU to do other things while waiting for a timeout, for example, polling other peripherals. It uses a really handy 32-bit counter that is clocked at 1 MHz; surprisingly, the same counter can be used on Linux, though in that case, you have to ask for permission from the OS to use it (e.g. using mmap).

To build the program, a makefile isn’t essential, it can be done with a single command line:

arm-none-eabi-gcc -specs=sdk/Alpha.specs -mfloat-abi=hard -mfpu=vfp -march=armv6zk -mtune=arm1176jzf-s -g3 -ggdb -Wall -Wl,-Tsdk/link.ld -Lsdk -Wl,-umalloc -o alpha_test.elf alpha_test.c

This produces the executable file alpha_test.elf. If your project involves other source files, they can be appended to the command line.

Running the program

Alpha provides the functionality of a remote gdb server, so we need to run a local instance of arm-none-eabi-gdb in remote mode. It is convenient to group all the gdb settings in a single file, named run.gdb:

source sdk/alpha.gdb
set serial baud 115200
target remote COM7
load
continue

On Linux, the serial port will be something like /dev/ttyUSB0 and you may need to set specific permissions for a user-mode program to access it.

We can now execute the code by running gdb with the settings, and executable filename:

arm-none-eabi-gdb -x run.gdb alpha_test.elf

If all is well, the code should load and run, flashing the ZeroW on-board LED at 1 Hz:

Loading section .entry, size 0x14f lma 0x8000
Loading section .text, size 0xaf00 lma 0x8150
Loading section .init, size 0x18 lma 0x13050
Loading section .fini, size 0x18 lma 0x13068
Loading section .rodata, size 0x310 lma 0x13080
Loading section .ARM.exidx, size 0x8 lma 0x13390
Loading section .eh_frame, size 0x4 lma 0x13398
Loading section .init_array, size 0x8 lma 0x2339c
Loading section .fini_array, size 0x4 lma 0x233a4
Loading section .data, size 0x9b0 lma 0x233a8
Start address 0x81c0, load size 48471
Transfer rate: 9 KB/sec, 850 bytes/write.

Alpha test............

Hit ctrl-C to halt the program, then ‘q’ to quit from GDB.

Unfortunately, if all is not well, there are no helpful error messages. If the target system is completely unresponsive, gdb will stall after the ‘reading symbols’ message; if it sees incorrect characters on the serial link it might report ‘a problem internal to GDB has been detected’; either way, the only option is to re-check the files on the SDHC card, and the serial connections.

Speedup

The upload is quite slow, so I wanted to speed it up. The limiting factor is the 115 kbaud serial speed, which is hard-coded into Alpha. However, gdb does have full access to all the on-chip registers, so it is possible to change the baud rate using GDB remote commands before downloading.

The commands have to be sent at 115 kbit/s to change the rate, and GDB must be reconfigured to use the serial link at the high speed. There are various ways this can be done; I decided to write a small program, alpha_speedup.py, that is compatible with Python 2.7 and 3.x:

# Utility for RPi Alpha to increase remote GDB baud rate
# From iosoft.blog, copyright (c) Jeremy P bentham 2020
# Requires pyserial package

import sys, serial, time

# Defaults
serport  = "COM7"
verbose  = False

# Settings
OLD_BAUD    = 115200
NEW_BAUD    = 921600
TIMEOUT     = 0.2
SYS_CLOCK   = 250e6

# BCM2835 UART baud rate divisor
uart_div    = int(round((SYS_CLOCK / (8 * NEW_BAUD)) - 1))

# GDB remote commands
high_speed  = "mw32 0x20215068 %u" % uart_div
qsupported  = "qSupported"

# Send command, return response
def cmd_resp(ser, cmd):
    txd = frame(cmd)
    if verbose:
        print("Tx: %s" % txd)
    ser.write(txd.encode('latin'))
    rxd = str(ser.read(1468))
    if verbose:
        print("Rx: %s" % rxd)
    resp = rxd.partition('$')
    return resp[2].partition('#')[0]

# Acknowledge a response
def ack_resp(ser):
    ser.write('+'.encode('latin'))
    if verbose:
        print("Tx: +")

# Return string, given hex values
def hex_str(hex):
    return bytearray.fromhex(hex).decode()

# Return remote hex command string
def cmd_hex(cmd):
    return "qRcmd,%s" % "".join([("%02x" % ord(c)) for c in cmd])

# Return framed data
def frame(data):
    return "$%s#%02x" % ("".join([escape(c) for c in data]), csum(data))

# Escape a character in the message
def escape(c):
    return c if c not in "#$}" else '}'+chr(ord(c)^0x20)

# GDB checksum calculation
def csum(data):
    return 0xff & sum([ord(c) for c in data])

# Open serial port
def ser_open(port, baud):
    try:
        ser = serial.Serial(port, baud, timeout=TIMEOUT)
    except:
        print("Can't open serial port %s" % port)
        sys.exit(1)
    return ser
        
# Close serial port
def ser_close(ser):
    if ser:
        ser.close()

if __name__ == "__main__":
    opt = None
    for arg in sys.argv[1:]:
        if len(arg)==2 and arg[0]=="-":
            opt = arg.lower()
            if opt == "-v":
                verbose = True
                opt = None
        elif opt == '-c':
            serport = arg
            opt = None
    print("Opening serial port %s at %u baud" % (serport, OLD_BAUD))
    ser = ser_open(serport, OLD_BAUD);
    cmd_resp(ser, "")
    ack_resp(ser)
    if cmd_resp(ser, qsupported):
        ack_resp(ser)
        print("Setting %u baud" % NEW_BAUD)
        cmd_resp(ser, cmd_hex(high_speed))
    time.sleep(0.01)
    print("Reopening at %u baud" % NEW_BAUD)
    ser_close(ser)
    ser = ser_open(serport, NEW_BAUD);
    ack_resp(ser)
    if cmd_resp(ser, qsupported):
        ack_resp(ser)
        print("Target system responding OK")
        time.sleep(0.01)
    else:
        print("No response from target system")
#EOF

For details of the commands, see the GDB remote specification. One unusual feature is that all responses from the target system have to be acknowledged with a ‘+’ character, otherwise they are re-sent 14 times. This is a bit awkward since the baud-rate change command acts immediately, so although it is sent at 115 kbit/s, the response is at 921600; we need to quickly close & reopen the port at the higher speed to send the acknowledgement.

I’ve hard-coded a Windows port (COM7) which will need to be changed for your setup, or use the command-line -c option to set something else (e.g. /dev/ttyUSB0). The -v option enables a verbose mode, that shows the commands and responses.

The second line of the GDB configuration file run.gdb needs to be changed to reflect the increased speed:

set serial baud 921600

The speedup program must always be run before gdb:

python alpha_speedup.py
arm-none-eabi-gdb -x run.gdb alpha_test.elf

Other features

Here are some things I’ve discovered about Alpha that might be useful:

ctrl-C: in my experimentation, you can only use ctrl-C to interrupt the program if it is printing to the console. There is presumably a way round this (apart from adding unnecessary print statements) but I don’t know what that is.

Console output: this works well, any print statements are echoed to the GDB console, but it does slow down the code a lot; if you need high speed, it is best to buffer the serial output in your program, then print it at the end.

GDB break: if you just want to quickly run a program, see the result, and exit GDB, this can be done by setting a breakpoint on a specific function, and setting an action when that is triggered, e.g. add the following lines to the end of run.gdb:

break gdb_break
commands 1
  kill
  quit
end

Now when the function ‘gdb_break’ is executed, GDB will exit back to the command-line. I add a matching dummy function to the C code:

// Dummy function to trigger gdb breakpoint
void gdb_break(void)
{
} // Trigger GDB break

..and just call this function if I want to halt the program and exit.

Copyright (c) Jeremy P Bentham 2020. Please credit this blog if you use the information or software in it.

Real Time Location using Ultra-Wideband (UWB)

Ultra-Wideband ranging modules

I’ve been looking for a system that can provide fast & accurate position measurement within a defined area; this is generally known as an RTLS (Real Time Location System).

My previous experiments have involved optical measurements, which have good accuracy, but are constrained by line-of-sight and range issues. So why not use wireless, and measure the time it takes for a radio pulse to travel from transmitter to receiver? Given the speed of light is roughly 300 mm (or 1 foot) per nanosecond, it may seem impossible to get an accurate position that way, but Decawave claim that their DW1000 time-of-flight chip gives measurements around 100 mm (4 inch) accuracy, using an Ultra Wideband (UWB) radio.

The DWM1000 module is bottom-left in the photo above, and consists of the DW1000 chip with a crystal reference, voltage regulator and on-board antenna. The other two boards use the same wireless chip, in different form-factors, but with additional embedded CPUs. I wanted to experiment with the DW1000 chip in a wide range of scenarios, and fully understand its low-level operation, so decided to use the simpler DWM1000 module, driven by my own Python code.

Decawave provide a large amount of documentation on their chip, and several software packages, but these are quite large: for example, their DecaRanging application has over 20,000 lines of C and C++ source code, which is a bit intimidating if you’re a newcomer to sub-nanosecond radio timing.

So I’ve created a Python test framework from scratch; at under 1000 lines of code, it can’t compete with the Decawave packages, but hopefully it’ll give an insight as to how the chip works, and enable you to experiment with this interesting technology.

Ultra Wideband

You may not have seen this RF technology before, but it has been around a while; the IEEE standard 802.15.4a is dated 2007. Just because it is part of the 802.15.4 family, you may think it is similar to Zigbee or 6LoWPAN, but that is not true. The RF operation is completely different: instead of transmitting on a single frequency, it covers a wide spectrum. This makes it much more resistant to single-frequency interferers, but of course raises the prospect that the UWB transmitter could interfere with other radio systems nearby.

For this reason, there are some quite complex rules about which frequencies can be used, the permissible power-levels, and the transmit repetition-rate. So it is possible that the RF emissions generated by my software are not permitted in your locality. If in doubt, consult a suitably-qualified RF engineer before doing any UWB testing.

Tags and Anchors

Real Time Location System

The standard Location System consists of several ‘anchors’, which are positioned at known locations, and ‘tags’ which move around a defined area; they determine their position by measuring the transit-time of the signals from several anchors.

This scheme works well for, say, locating people within a shopping mall; their mobile phones can be the tags, displaying the location within that mall – and non-coincidentally, the Apple iPhone 11 does have an UWB capability.

An alternative scenario is where the tags are fitted to vehicles in a warehouse, allowing a management system to track their whereabouts. There are two ways this can be achieved; either a tag just transmits a simple beacon message, and the anchors share their time-measurements to establish its position, or alternatively the tag measures its distance from the nearest anchors, and transmits the result for forwarding to the management system.

Implicitly, a tag is a battery powered device that only transmits occasionally, but in reality there are many other ways to configure a location network, depending on the overall requirements.

This flexibility comes from the fact that the ranging messages can also carry data (up to 127 bytes as standard), so there are numerous ways the RTLS can be structured. In this first post, I’m ignoring all that complexity, and just focusing on the distance measurement between two systems, which could be tags, anchors, or anything else you decide.

Ranging

Ranging is the process whereby two UWB radio systems can measure the distance between themselves. Simplistically, one might think that it is just necessary for the transmitter to note the time of a message transmission, and the receiver to note the time it is received: subtract the two and you get a time-difference, which is directly proportional to the distance between them.

However, it isn’t quite that simple, because:

  1. The measurement has to be very accurate; a radio wave travels at around 300,000,000 metres per second (1 foot per nanosecond) so to achieve any degree of accuracy, we need a time measurement in picoseconds (10-12 seconds).
  2. In this method, the time-clocks of the transmitter and receiver have to be very accurately synchronised, and that isn’t easy.
  3. To keep hardware costs down, each of the units will have an inexpensive quartz crystal as the timing reference, and we have to accommodate variations in the crystal frequency due to its tolerance, temperature drift, etc.
  4. The radio wave that arrives at the receiver won’t be an accurate copy of what is transmitted; there will be distortions due to the radio circuitry, and reflections from nearby objects.

Fortunately, these problems can be addressed by using a technique called ‘Asymmetric Two Way Ranging’:

  1. Use very fast, high-resolution timers; the sampling clock on the the DW1000 runs at 63.8976 GHz, and feeds a 40-bit counter.
  2. Don’t synchronise the clocks in the transmitter and receiver; let them just free-run.
  3. Measure the difference in crystal frequency, and compensate for it.
  4. Use Ultra-Wideband (UWB) which is more resilient than conventional radio systems.
Asymetric Two-Way Ranging

In the diagram above, there are 3 messages passing between two units; unit A transmits messages 1 and 3, unit B sends message 2. Each unit records a timestamp when the message was sent or received, so we have a total of 6 timestamps, from which to determine the transit time, and hence the distance.

Simplistically the transit time can be measured by comparing Rx2 – Tx1 with Tx2 – Rx1, but you’ll see that the time clocks for units A and B are running at different speeds. In reality they’ll only differ by a few parts per million (the difference has been greatly magnified for the illustration) but a small difference creates in a large position error, so we need a method to compensate for it. This is done by getting the two units to make the same measurement, and comparing the result; the obvious candidate is the time between the two transmissions (Tx3 – Tx1) and the time between the two receptions (Rx3 – Rx1). These should be equal, so the ratio of the times will be the ratio of their clock frequencies.

The final formula for the transit time (taken from Decwave’s APS013 application note) is:

rnd1 = Rx2 - Tx1 # Round-trip 1 to 2
rep1 = Tx2 - Rx1 # Reply time 1 to 2
rnd2 = Rx3 - Tx2 # Round-trip 2 to 3
rep2 = Tx3 - Rx2 # Reply time 2 to 3 
time = (rnd1 × rnd2 - rep1 * rep2) / (rnd1 + rnd2 + rep1 + rep2) 

Hardware

DWM1000 module on carrier board

The Decawave DW1000 chip can be purchased from electronic distributors, but unless you’re into microwave PCB design, you’ll want to buy a pre-packaged module. The simplest of these is the DWM1000, which includes the necessary power circuitry and ceramic chip antenna. It has no on-board CPU, so is driven by an external processor over a 4-wire Serial Peripheral Interface (SPI).

You could solder wires direct to the package, but I used an adaptor board that brings out the connections to a breadboard-friendly 0.1″ pitch. The adaptor is the “DWM1000 Breakout-01”, available from OSH Park.

Aside from the SPI interface (CLK, MISO, MOSI and CS) you only have to provide 3.3V power and ground, though I also connected reset (RST) and interrupt (IRQ) signals. Reset is very useful to clear down the chip before programming, and the interrupt saves the chip from frenetic polling during transmission or reception (which can disrupt the RF section of some chips).

Which CPU to drive the module? Any microcontroller would do, but I’d like to control the modules using a single Python program on a PC; this is much easier than updating multiple copies of the software on different CPUs. So I’m attaching each module to a Raspberry Pi, to act as a relatively dumb network-to-SPI converter; I can then send streams of SPI commands from the PC program over a WiFi network to 2 or more UWB modules, without having to reprogram their CPUs.

Pi ZeroW and DWM1000 module

SPI port 0 or 1 can be used on the RPi, so long as it is enabled in /boot/config/txt. The pin numbers are:

# Connector pin numbers:
#       SPI0        SPI1
# GND   25          34
# CS    24 (CE0)    36 (CE2)
# MOSI  19          38
# MISO  21          35
# CLK   23          40
# IRQ   18          32
# RESET 22 (BCM25)  37 (BCM26)
# NRST  16 (BCM23)  31 (BCM6) 
# 3.3V  17

I have provided a positive-going reset signal (RESET) and negative-going (NRST). This is because my early hardware had a transistor inverter in the reset line, so needed a positive-going signal. If you are connecting the RPi pin direct to the module, use the NRST signal. [And in case you’re wondering, I realise that the RPi mustn’t drive the module reset line high; my software does not do this, it drives the line low to reset, or lets it float.]

A useful extra is to fit an LED indicator to the module interrupt line (with a current-limiting resistor of a few hundred ohms to ground). This will flash in a recognisable pattern when ranging is working correctly, which is very useful when testing the module’s operational limits.

The module with a Pi ZeroW and USB power pack is a neat package; I had some concerns about taking 3.3V power from the RPi, due to possible electrical noise issues, but it seems to work fine, providing you keep the cable to the module short – I’d suggest a maximum of 100 mm (4 inches) if you want to avoid problems.

Raspberry Pi Software

Ranging test system

We need a simple way of sending commands to the network nodes from the PC; since each command is a small data block, and we have to wait for the command to be executed before sending the next, the logical choice is User Datagram Protocol (UDP). This is an ‘unreliable’ protocol, as it has no mechanism for retrying any lost transmissions, or eliminating any duplicates, so I’ve added a lightweight client-server error-handling layer. Each data block (‘datagram’ in UDP parlance) has a 1-byte sequence number, a 1-byte length, and a payload of up to 255 bytes. The client (PC system) increments the sequence number with each new transmission; the server (Raspberry Pi) checks whether that sequence number has already been received. If so, the data is ignored, and the previous response is just resent; if not, the data is sent to the UWB module over the SPI interface, and the response is returned to the client.

Network Server

The code on the Raspberry Pi has been kept simple; it is single-threaded by using the ‘select’ mechanism to poll the socket for incoming data, with a timeout that allows the interrupt indicator to be polled:

import socket, select

# Simple UDP server
class Server(object):
    def __init__(self):
        self.rxdata, self.txdata = [], []
        self.sock = self.addr = None

    # Open socket
    def open(self, portnum):
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        self.sock.bind(('', portnum))
        return self.sock

    # Receive incoming data with timeout
    def recv(self, maxlen=MAXDATA, timeout=SOCK_TIMEOUT):
        rxdata = []
        socks = [self.sock]
        rd, wr, ex = select.select(socks, [], [], timeout)
        for s in rd:
            rxdata, self.addr = s.recvfrom(maxlen)
        return rxdata

    # Receive incoming request, return iterator for data blocks
    # If sequence number is unchanged, resend last transmission
    def receive(self):
        self.rxdata = bytearray(self.recv(MAXDATA))
        if len(self.rxdata) > SEQLEN:
            if len(self.txdata)>SEQLEN and self.rxdata[0]==self.txdata[0]:
                self.xmit(self.txdata)
            else:
                self.txdata = [self.rxdata[0]]
                rxd = self.rxdata[SEQLEN-1:]
                while len(rxd)>1 and len(rxd)>rxd[0]:
                    n = rxd[0] + 1
                    yield(rxd[1:n])
                    rxd = rxd[n:]

    # Add response data to list
    def send(self, data):
        self.txdata += [len(data)] + data

    # Transmit responses
    def xmit(self, txdata, suffix=''):
        if self.addr and len(txdata)>SEQLEN:
            txd = bytearray(txdata)
            self.sock.sendto(txd, self.addr)

SPI interface

This consists of a clock line, data from the RPi to the module (MOSI: Master Out Slave In), data from the module (MISO: Master In Slave Out) and a Chip Select (CS) line that frames each transmission.

For protocol details, see the Decawave DW1000 datasheet. The most significant bit of the first byte indicates a read or write cycle; a read cycle returns one or more garbage bytes (depending on the addressing mode) followed by the actual data; my software returns all of these bytes back to the PC. A write-cycle returns no useful data (it is generally all-ones) but this is still passed back to the PC, as an acknowledgement that the write command has been received.

import spidev, RPi.GPIO as GPIO

# Open SPI interface
spif = 0,0
spi = spidev.SpiDev()
spi.open(*spif)
spi.max_speed_hz = 2000000
spi.mode = 0

# Set up board I/O
rst_pin, irq_pin = 22, 18
GPIO.setmode(GPIO.BOARD)
GPIO.setwarnings(False)
GPIO.setup(rst_pin, GPIO.OUT)
GPIO.setup(irq_pin, GPIO.IN)
GPIO.add_event_detect(irq_pin, GPIO.RISING, callback=irq_handler)

The clock speed of 2 MHz is well within the specified limits for the module. The interrupt (IRQ) line is high when asserted, so positive-edge-detection is used; the callback just sets a global variable that is polled in the main loop.

Running code on startup

It is convenient for the SPI server code to automatically run when the RPi boots; there are various ways to do this, which are beyond the scope of this blog. I used systemd as follows:

sudo systemctl edit --force --full spi_server.service

# Add the following to spi_server.service..
 [Unit]
   Description=SPI server
   Wants=network-online.target
   After=network-online.target
 [Service]
   Type=simple
   User=pi
   WorkingDirectory=/home/pi/uwb
   ExecStart=/usr/bin/python spi_server.py
 [Install]
   WantedBy=multi-user.target

# Enable the service using:
sudo systemctl enable spi_server.service
sudo systemctl start spi_server.service  # ..or 'stop' to stop it

# To check if service is running..
systemctl status spi_server

Main Program

This Python program (dw1000_range.py) runs on a PC, feeding command strings over the network to the Raspberry Pi UDP-to-SPI adaptors.

Device Initialisation

The bulk of the main program is involved in device initialisation, as the DW1000 has a remarkably large number of registers – my software defines 106, and that isn’t all of them. To add to the complexity, they vary in size between 1 and 14 bytes, have multiple bitfields within them, and are accessed by a multi-level addressing scheme.

By any measure, this is a complex chip, and is a very easy for the software to degenerate into endless sequences of ANDing SHIFTing and ORing to insert new data into a register. To avoid this, the C language has bitfields, and the equivalent in Python is ‘ctypes’, indeed this library was created to allow Python to access DLLs written in C.

I’ve used ctypes in a novel way to give a clean way of reading & writing one or more fields of a register, without any cumbersome logic operations.

To give a simple example, DW1000 register 0 is 32 bits wide, containing a 4-bit revision number in the least significant bits, then a 4-bit version, 8-bit model, and a 16-bit tag number.

I have defined this as:

DEV_ID = 0x0, 4, None, (("REV", U32, 4), ("VER",   U32, 4),
                        ("MODEL",U32, 8), ("RIDTAG",U32,16))

This data is passed to a Python class:

from ctypes import LittleEndianStructure as Structure, Union
from ctypes import c_uint as U32, c_ulonglong as U64

# DW1000 register class
class Reg(object):
    def __init__(self, regdef, val=0):
        self.name, self.value = regdef, val
        self.id, self.len, self.sub, self.fields = globals()[regdef]
        class struct(Structure):
            _fields_ = self.fields
        class union(Union):
            _fields_ = [("reg", struct), ("value", U64)]
        self.u = union()
        self.u.value = val
        self.reg = self.u.reg

    # Read register value
    def read(self, spi):
        # [Do SPI read cycle]
        return self

    # Write register value
    def write(self, spi):
        # [Do SPI write cycle]
        return self

# Set a field within a register
    def set(self, field, val):
        if hasattr(self.reg, field):
            setattr(self.reg, field, val)
        else:
            print("Unknown attribute: '%s'" % field)
        self.value = self.u.value
        return self

The union overlays an array of bytes on top of the register value; this provides a byte data-stream to be used by the SPI read & write functions.

Instantiating the class gives us a local copy of the DW1000 register, and the ‘read’ method populates the copy with values from the remote register, e.g.

r = Reg('DEV_ID')
r.read(spi)
print("%X" % r.reg.RIDTAG)

Note that the class methods return ‘self’, so can be chained; for example, here is a read-modify-write cycle that sets the transmit frame length, which is in the bottom 7 bits of the 40-bit register 8:

TX_FCTRL  = 0x8, 5, None,(("TFLEN", U64, 7), ("TFLE", U64, 3), ("R", U64, 3),
                          ("TXBR",  U64, 2), ("TR",   U64, 1), ("TXPRF", U64, 2),
                          ("TXPSR", U64, 2), ("PE",   U64, 2), ("TXBOFFS", U64, 10),
                          ("IFSDELAY",  U64, 8))

Reg('TX_FCTRL').read(spi).set('TFLEN', txlen).write(spi)

‘spi’ in these examples is a class instance that contains the code to read or write the SPI interface; in my test framework, this is actually a network interface that sends the data to a Raspberry Pi, and obtains the response. This is necessary because I have one Python program controlling two (or more) DW1000 modules, so I need a class instance for each SPI interface, giving an IP address and UDP port number, e.g.

# Class for an SPI interface
class Spi(object):
    def __init__(self, spif, ident='1'):
        self.spif, self.ident = spif, ident
        self.txseq = 0
        self.verbose = self.interrupt = False
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        if self.sock:
            self.sock.connect(spif[1:])
            print("Connected to %s:%u" % spif[1:])
        else:
            print("Can't open socket")

SPIF = "UDP", "10.1.1.226", 1401
print("Connecting to %s port %s:%u" % SPIF)
spi = Spi(SPIF)

Before leaving the subject of device initialisation, it is worth mentioning that I’ve used several Python dictionaries (as lookup tables) to simplify the underlying logic: for example, the transmitter analog setting RF_TXCTRL, which depends on the channel number (1 – 7, excluding 6)

CHAN_RF_TXCTRL  = {1:0x5c40, 2:0x45ca0, 3:0x86cc0, 4:0x45c80, 5:0x1e3fe0, 7:0x1e7de0}
Reg('RF_TXCTRL', CHAN_RF_TXCTRL[chan]).write(self.spi)

The register class is instantiated using a value from the dictionary, then that value is written to the hardware.

There is a drawback to my approach; the maximum size of any register is limited to 64 bits (c_ulonglong). Fortunately there are only 2 registers longer than this (RX_TIME: 14 bytes, TX_TIME: 10 bytes) and these can be split into sections to come within the 8-byte limit.

Frame Format

A transmitted message (‘frame’) consists of a preamble that the receiver will synchronise to, a start-of-frame delimiter (SFD), and the data payload. The preamble & SFD are automatically inserted by the DW1000, and are used by the timing logic to produce an accurate timestamp, so generally the recommended values will be used. The data payload, however, can be anything; if you want to inter-operate with other UWB 802.15.4 devices it can be a maximum of 127 bytes and must have a standardised header; if not, it can be any format up to 1023 bytes.

Normally, the payload would be used to convey timing information from a tag to an anchor, but in my case the main Python program has visibility of all data through the Wifi network, so I don’t need to send any data across UWB. Arbitrarily, I chose to send the data of an 802.15.4 ‘blink’, which is a very short message containing a 1-byte prefix, 1-byte sequence number and 8-byte address.

# Blink frame with IEEE EUI-64 tag ID
BLINK_FRAME_CTRL = 0xc5
BLINK_MSG=(('framectrl',   U8), 
           ('seqnum',      U8),
           ('tagid',       U64))

This is instantiated using a Frame class that is similar to the Reg class described above, allowing us to refer to the fields individually, or collectively as a stream of bytes.

blink1 = Frame(BLINK_MSG)
blink1.values.framectrl = BLINK_FRAME_CTRL
blink1.values.tagid = 0x0101010101010101
txdata = blink1.data()

Transmission

Having done all the hard work initialising the chip, transmission is just a question of setting the frame data & length, then setting a single bit.

dw1 = DW1000(spi1)
dw1.initialise()
dw1.set_txdata(txdata)
dw1.start_tx()

The timing-specific information is handled automatically, so the precise time of the transmission (specifically, the timing of the SFD) can be determined by a single function call:

    # Get Tx timestamp
    def tx_time(self):
        return Reg('TX_TIME1').read(self.spi).reg.TX_STAMP

You can set the hardware to generate an interrupt (IRQ signal) when transmission is complete, but I haven’t found this necessary.

Reception

To receive a frame, the preamble, SFD and data must be decoded; the data must pass a CRC check, and the address must match the filtering criteria if these are enabled. Success or failure is signalled by various bits in the SYS_STATUS register; these bits can also be used to signal an interrupt, if enabled in the SYS_MASK register. In my code, the following signals are enabled as interrupts:

  • RXPHE: phy header error
  • RXFCG: receiver frame check good
  • RXFCE: receiver frame check eror
  • RXRFSL: receiver frame sync loss
  • RXRFTO: receiver frame wait timeout
  • RXSFDTO: receive SFD timeout
  • AFFREJ: automatic frame filtering reject

The most important signals are RXFCG / RXFCE to signal a good frame, or an error condition. The Decawave code has complex error handling, to tackle some ways in which the chip might lock up, and stop responding. Since we have one master program controlling both transmission and reception, we can adopt a simplistic approach to error handling, and if a chip fails to receive several consecutive transmissions, it is reset and re-initialised.

Assuming reception is successful, the data and timestamp can be read out; in our case, we’re only really interested in the time:

    # In DW1000 class..
    def get_rxdata(self):
        rxdata = []
        if self.check_interrupt():
            status = Reg('SYS_STATUS').read(self.spi)
            if status.reg.RXDFR:
                rxdata = self.rx_data()
        return rxdata

    # Get Rx timestamp
    def rx_time(self):
        return Reg('RX_TIME1').read(self.spi).reg.RX_STAMP
...
rxdata = dw1.get_rxdata()
dt1 = dw1.rx_time() - dw1.tx_time()
dt2 = dw2.tx_time() - dw2.rx_time()

Running the test

The Python source files are on Github, they are:

Main PC program:

  • dw1000_range.py: main program to run the test
  • dw1000_regs.py: classes describing the UWB chip internals
  • dw1000_spi.py: SPI-over-UDP interface

Raspberry Pi:

  • spi_server.py

The files are compatible with Python 2.7 or 3.x

I didn’t get around to providing a neat UI on the main program, so at the top of dw1000_range.py you have to enter the IP addresses of the two RPi units, e.g.

# Specify SPI interfaces:
#   "UDP", "<IP_ADDR>", <PORT_NUM>
SPIF1       = "UDP", "10.1.1.235", 1401
SPIF2       = "UDP", "10.1.1.230", 1401

There is an optional verbose ‘-v’ command-line flag that enables the display of all the incoming & outgoing data. This includes a modulo-10-second timestamp with 1 millisecond resolution, which is useful for tracking down timing problems.

The SPI server running on the RPi units has a similar ‘-v’ option for verbose mode, and an optional port number that also changes the SPI interface, so port 1401 can be SPI0, and 1402 can be SPI1.

To run the test, first make sure the SPI servers are running on the 2 RPis; you can run them from ssh consoles, but I have found that this noticeably degrades the UDP response times on a Pi Zero (see the ‘potential problems’ section below) so once you’ve proved they work, I’d recommend running the code on startup, as described above.

Then run the main program; you should see a stream of ranging results, e.g.

Connected to 10.1.1.226:1401
Connected to 10.1.1.230:1401
147.136 156.569
146.991 156.616
147.127 156.602
146.053 156.555
144.017 156.555
146.001 156.588 
..and so on..

This is from 2 units 2 metres (6.5 feet) apart. The first column is the distance in metres for simple 2-message ranging with no measurement of the difference in clock frequencies; the second column is for full asymmetric ranging, that uses a total of 3 messages to compensate for clock inaccuracies.

I’ve said that the units are 2 metres apart, so you’d expect a value of 2 to be displayed, not 147 or 156. The reason for this discrepancy is that the RF circuitry adds a very large time-delay to the signal, that has to be subtracted from the final result. The best way to calculate this compensation value is to measure several known distances, and adjust the multiplier and constant values to produce the right answers.

I haven’t done this calibration process yet, so the un-adjusted result is displayed. The main focus of my current test is to see how repeatable the results are, i.e. how much jitter there is in the position value.

Taking 1000 readings, at distances between 2 and 6 metres, (roughly 6.5 to 20 feet) produces the following histogram of the error between the actual distance (as indicated by the average of all the samples) and the reported distance:

You’ll see the error doesn’t get much greater as the distance increases, i.e. it is not a percentage of the distance measured. This shows that (under good-signal conditions) the main error source is the jitter in the capture and measurement of the incoming wave, as discussed in the Decawave literature, and this is relatively constant irrespective of distance.

The above tests are in good line-of-sight conditions, so to degrade the signal I did a 9 metre (30 foot) range test obstructed by a sizeable brick wall (1900-vintage, not a flimsy modern partition) and a few items of furniture. To my surprise, the error histogram didn’t show very much degradation.

It is also worth noting that despite the convoluted control and measurement process (PC talking to Raspberry Pis), around 5 ranging results are returned per second. Using local CPUs (as in many of the Decawave demonstration systems) will produce a major speed improvement, and a simple rolling average would markedly improve the position accuracy.

Potential problems

Here are some issues you may encounter:

  1. Power supply. In my experience, the most common problem is with the power supply. When receiving or transmitting, the Decawave module takes around 160 milliamps, which is more than some simple 3.3V supplies can handle. Also, the module may appear to work, even if the power supply is completely disconnected; the startup current is sufficiently low that the module can power itself from the I/O lines, and return a valid ID across the SPI interface, even though it is unpowered. Of course it will fail as soon as any real operations start, but the initial SPI response may lead you to look for complex bugs in your code, rather than a simple power supply fault.
  2. IRQ. The software does include a check that the interrupt (IRQ) line is operational, by setting it as an output, then toggling it; see the ‘pulse_irq’ method. If this check fails, there is no point proceeding with the tests.
  3. Missed interrupts. After each transmission, the main software waits for 50 milliseconds to get an interrupt from the receiving unit; if this doesn’t arrive, it polls the receiver’s status register, and if an interrupt is pending (i.e. the message has been received), it reports ‘missed interrupt’. This is harmless, and could be disabled; the reason is that the Raspberry Pi networking stack occasionally adds a long delay to UDP transmissions. To see this in action, try sending flood pings from the RPi to a fast server; I’ve seen round-trip times vary between 3 and 80 msec, even if the WiFi network is very lightly loaded, and the RPi is doing nothing else.
  4. Hardware Reset. Whilst not essential for the testing, if the reset line isn’t connected, things can get confusing, since the chip won’t be cleared down between tests, and the software does assume a ‘clean slate’ at the start of the test.
  5. Status display. If reception fails with error flags set, I display the receiver status; this information is useful in formulating an error-handling strategy.
  6. Bursts of failures. Sometimes when seeing a poor signal, the units stop communicating, and rack up continuous errors. If my software detects 10 such errors, it resets the two units, then carries on as normal. This is not the correct approach; if you look at the Decawave source code, they check the status register to look for potential lock-up conditions, and take appropriate action; they don’t wait for multiple failures.
  7. RF performance. Another weakness of my approach is that it doesn’t represent an accurate simulation of the RF performance of the Decawave chips. Radio circuitry needs decent RF design, and putting a module with an adaptor PCB on a breadboard is not good from an RF perspective. It is credit to the Decwave designers that their module tolerates this approach, producing reasonable results – but if they fall short of expectations, don’t be too surprised; to get the best from the chips and modules, proper hardware design is required.

Ultra-Wideband is a remarkable technology, and I’ve only scratched the surface of what it can do; now see what you can make of it…

Copyright (c) Jeremy P Bentham 2019. Please credit this blog if you use the information or software in it.

ARM GCC Lean: programming and debugging the Nordic NRF52

The nRF52832 is an ARM Cortex M4 chip with an impressive range of peripherals, including an on-chip 2.4 GHz wireless transceiver. Nordic supply a comprehensive SDK with plenty of source-code examples; they are fully compatible with the GCC compiler, but there is little information on how to program and debug a target system using open-source tools such as the GDB debugger, or the OpenOCD JTAG/SWD programmer.

This blog will show you how to compile, program and debug some simple examples using the GNU ARM toolchain; the target board is the NRF52832 Breakout from Sparkfun, and the programming is done via a Nordic development board, or OpenOCD on a Raspberry Pi. Compiling & debugging is with GCC and GDB, running on Windows or Linux.

Source files

All the source files are in an ‘nrf_test’ project on GitHub; if you have Git installed, change to a suitable project directory and enter:

git clone https://github.com/jbentham/nrf_test

Alternatively you can download a zipfile from github here. You’ll also need the nRF5 15.3.0 SDK from the Nordic web site. Some directories need to be copied from the SDK to the project’s nrf5_sdk subdirectory; you can save disk space by only copying components, external, integration and modules as shown in the graphic above.

Windows PC hardware

Cortex Debug Connection to a Nordic evaluation board.

The standard programming method advocated by Nordic is to use the Segger JLink adaptor that is incorporated in their evaluation boards, and the Windows nRF Command Line Tools (most notably, the nrfjprog utility) that can be downloaded from their Web site.

Connection between the evaluation board and target system can be a bit tricky; the Sparkfun breakout board has provision for a 10-way Cortex Debug Connector, and adding the 0.05″ pitch header does require reasonable soldering skills. However, when that has been done, a simple ribbon cable can be used to connect the two boards, with no need to change any links or settings from their default values.

One quirk of this arrangement is that the programming adaptor detects the 3.3V power from the target board in order to switch the SWD interface from the on-board nRF52 chip to the external device. This has the unfortunate consequence that if you forget to power up the target board, you’ll be programming the wrong device, which can be confusing.

The JLink adaptor isn’t the only programming option for Windows; you can use a Raspberry Pi with OpenOCD installed…

Raspberry Pi hardware

Raspberry Pi SWD interface (pin 1 is top right in this photo)

In a previous blog, I described the use of OpenOCD on the raspberry Pi; it can be used as a Nordic device programmer, with just 3 wires: ground, clock and data – the reset line isn’t necessary. The breakout board needs a 5 volt supply which could be taken from the RPi, but take care: accidentally connecting a 5V signal to a 3.3V input can cause significant damage.

Rasberry Pi SWD connections
NRF52832 breakout SWD connections

Install OpenOCD as described in the previous blog; I’ve included the RPi and SWD configuration files in the project openocd directory, so for the RPi v2+, run the commands:

cd nrf_test
sudo openocd -f openocd/rpi2.cfg -f openocd/nrf52_swd.cfg

The response should be..

BCM2835 GPIO config: tck = 25, tms = 24, tdi = 23, tdo = 22

Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : BCM2835 GPIO JTAG/SWD bitbang driver
Info : JTAG and SWD modes enabled
Info : clock speed 1001 kHz
Info : SWD DPIDR 0x2ba01477
Info : nrf52.cpu: hardware has 6 breakpoints, 4 watchpoints
Info : Listening on port 3333 for gdb connections

The DPIDR value of 0x2BA01477 is correct for the nRF52832 chip; if any other value appears, there is a problem: check the wiring.

Windows development tools

The recommended compiler toolset for the SDK files is gcc-arm-none-eabi, version 7-2018-q2-update, available here. This places the tools in the directory

C:\Program Files (x86)\GNU Tools Arm Embedded\7 2018-q2-update\bin

Check that this directory in included in your search path by opening a command window, and typing

arm-none-eabi-gcc  -v

If not found, close the window, add to the PATH environment variable, and retry.

You will also need to install Windows ‘make’ from here. At the time of writing, the version is 3.81, but I suspect most modern versions would work fine. As with GCC, check that it is included in your executable path by opening a new command window, and typing

make -v

Linux development tools

A Raspberry Pi 2+ is quite adequate for compiling and debugging the test programs.

Although RPi Linux already has an ARM compiler installed, the executable programs it creates are heavily dependant on the operating system, so we also need to install a cross-compiler: arm-none-eabi-gcc version 7-2018-q2-update. The easiest way to do this is to click on Add/Remove software in the Preferences menu, then search for arm-none-eabi. The correct version is available on Raspbian ‘Buster’, but probably not on earlier distributions.

The directory structure is the same as for Windows, with the SDK components, external, integration and modules directories copied into the nrf5_sdk subdirectory.

As with Windows, it is worth typing

arm-none-eabi-gcc  -v

..to make sure the GCC executable is installed correctly.

nrf_test1.c

This is in the nrf_test1 directory, and is as simple as you can get; it just flashes the blue LED at 1 Hz.

// Simple LED blink on nRF52832 breakout board, from iosoft.blog

#include "nrf_gpio.h"
#include "nrf_delay.h"

// LED definitions
#define LED_PIN      7
#define LED_BIT      (1 << LED_PIN)

int main(void)
{
    nrf_gpio_cfg_output(LED_PIN);

    while (1)
    {
        nrf_delay_ms(500);
        NRF_GPIO->OUT ^= LED_BIT;
    }
}

// EOF

An unusual feature of this CPU is that the I/O pins aren’t split into individual ports, there is just a single port with a bit number 0 – 31. That number is passed to an SDK function to initialise the LED O/P pin, and I could have used another SDK function to toggle the pin, but instead used an exclusive-or operation on the hardware output register.

The SDK delay function is implemented by performing dummy CPU operations, so isn’t particularly accurate.

Compiling

For both platforms, the method is the same: change directory to nrf_test1, and type ‘make’; the response should be similar to:

Assembling ../nrf5_sdk/modules/nrfx/mdk/gcc_startup_nrf52.S
 Compiling ../nrf5_sdk/modules/nrfx/mdk/system_nrf52.c
 Compiling nrf_test1.c
 Linking build/nrf_test1.elf
    text    data     bss     dec     hex filename
..for Windows..
    1944     108      28    2080     820 build/nrf_test1.elf
..or for Linux..
    2536     112     172    2820     b04 build/nrf_test1.elf

If your compile-time environment differs from mine, it shouldn’t be difficult to change the Makefile definitions to match, but there are some points to note:

  • The main changeable definitions are towards the top of the file. Resist the temptation to rearrange CFLAGS or LNFLAGS, as this can create a binary image that crashes the target system.
  • You can add files to the SRC_FILES definition, they will be compiled and linked in; the order of the files isn’t significant, but I generally put gcc_startup_nrf52.S first, so Reset_Handler is at the start of the executable code. Similarly, INC_FOLDERS can be expanded to include any other folders with your .h files.
  • The task definitions toward the bottom of the file use the tab character for indentation. This is essential: if replaced with spaces, the build process will fail.
  • ELF, HEX and binary files are produced in the ‘build’ subdirectory; ELF is generally used with GDB, while HEX is required by the JLink flash programmer.
  • I’ve defined the jflash and ocdflash tasks, that do flash programming after the ELF target is built; you can add your own custom programming environment, using a similar syntax.
  • The makefile will re-compile any C source files after they are changed, but will not automatically detect changes to the ‘include’ files, or the makefile itself; when these are edited, it will be necessary to force a re-make using ‘make -B’.
  • If a new image won’t run on the target system, the most common reason is an un-handled exception, and it can be quite difficult to find the cause. So I’d recommend that you expand the code in relatively small steps, making it easier to backtrack if there is a problem.

Device programming

Having built the binary image, we need to program it into Flash memory on the target device. This can be done by:

  • JLink adaptor on an evaluation board (Windows PC only)
  • Directly driving OpenOCD (RPi only)
  • Using the GNU debugger GDB to drive OpenOCD (both platforms)

Device programming using JLink

Set up the hardware and install the Nordic nRF Command Line Tools as described above, then the nrfjflash utility can be used to program the target device with a hex file, e.g.

nrfjprog --program build/nrf_test1.hex --sectorerase
nrfjprog --reset

The second line resets the chip after programming, to start the program running. This is done via the SWD lines, a hardware reset line isn’t required; alternatively you can just power-cycle the target board.

The above commands have been included in the makefile, so if you enter ‘make jflash’, the programming commands will be executed after the binary image is built.

An additional usage of the JLink programmer is to restore the original Arduino bootloader, that was pre-installed on the Sparkfun board. To do this, you need to get hold of the softdevice and DFU files from the Sparkfun repository, combine them using the Nordic merge utility, then program the result using a whole-chip erase:

mergehex -m s132_nrf52_2.0.0_softdevice.hex sfe_nrf52832_dfu.hex -o dfu.hex
nrfjprog --program dfu.hex --chiperase
nrfjprog --reset 

Device programming using OpenOCD

OpenOCD can be used to directly program the target device, providing the image has been built on the Raspberry Pi, or the ELF file has been copied from the development system. Install and test OpenOCD as described in the Raspberry Pi Hardware section above (check the DPIDR value is correct), hit ctrl-C to terminate it, then enter the command:

sudo openocd -f ../openocd/rpi2.cfg -f ../openocd/nrf52_swd.cfg -c "program build/nrf_test1.elf verify reset exit"

The response should be similar to:

 ** Programming Started **
 Info : nRF52832-QFAA(build code: E0) 512kB Flash
 Warn : using fast async flash loader. This is currently supported
 Warn : only with ST-Link and CMSIS-DAP. If you have issues, add
 Warn : "set WORKAREASIZE 0" before sourcing nrf51.cfg/nrf52.cfg to disable it
 ** Programming Finished **
 ** Verify Started **
 ** Verified OK **
 ** Resetting Target **
 shutdown command invoked

Note the warnings: by default, OpenOCD uses a ‘fast async flash loader’ that achieves a significant speed improvement by effectively sending a write-only data stream. Unfortunately the Nordic chip occasionally takes exception to this, and returns a ‘wait’ response, which can’t be handled in fast async mode, so the programming fails – in my tests with small binary images, it does fail occasionally. As recommended in the above text, I’ve tried adding ‘set WORKAREASIZE 0’ to nrf52_swd.cfg (before ‘find target’), but this caused problems when using GDB. By the time you read this, the issue may well have been solved; if not, you might have to do some experimentation to get reliable programming.

The makefile includes the OpenOCD direct programming commands, just run ‘make ocdflash’.

Device programming using GDB and OpenOCD

The primary reason for using GDB is to debug the target program, but it can also serve as a programming front-end for OpenOCD. This method works with PC host, or directly on the RPi, as shown in the following diagram.

GDB OpenOCD debugging

In both cases we are using the GB ‘target remote’ command; on the development PC we have to specify the IP address of the RPi: for example, 192.168.1.2 as shown above. If in doubt as to the address, it is displayed if you hover the cursor over the top-right network icon on the RPi screen. By default, OpenOCD only responds to local GDB requests, so the command ‘bindto 0.0.0.0’ must be added to the configuration. This means anyone on the network could gain control of OpenOCD, so use with care: consider the security implications.

Alternatively, the Raspberry Pi can host both GDB and OpenOCD, in which case the ‘localhost’ address is used, and there is no need for the additional ‘bindto’.

The commands for the PC-hosted configuration are:

# On the RPi:
  sudo openocd -f ../openocd/rpi2.cfg -f ../openocd/nrf52_swd.cfg -c "bindto 0.0.0.0"

# On the Windows PC:
arm-none-eabi-gdb -ex="target remote 192.168.1.2:3333" build\nrf_test1.elf -ex "load" -ex "det" -ex "q"

The PC connects to the OpenOCD GDB remote server on port 3333, loads the file into the target flash memory, detaches from the connection, and exits. The response will be something like:

Loading section .text, size 0x790 lma 0x0
 Loading section .ARM.exidx, size 0x8 lma 0x790
 Loading section .data, size 0x6c lma 0x798
 Start address 0x2b4, load size 2052
 Transfer rate: 4 KB/sec, 684 bytes/write.
 Detaching from program: c:\Projects\nrf_test\nrf_test1\build\nrf_test1.elf, Remote target
 Ending remote debugging.

I have experienced occasional failures with the message “Error finishing flash operation”, in which case the command must be repeated; see my comments on the ‘fast async flash loader’ above.

The Rpi-hosted command sequence is similar:

# On the RPi (first terminal):
sudo openocd -f ../openocd/rpi2.cfg -f ../openocd/nrf52_swd.cfg

# On the RPi (second terminal):
gdb -ex="target remote localhost" build\nrf_test1.elf -ex "load" -ex "det" -ex "q" 

Note that the GDB programming cycle does not include a CPU reset, so to run the new program the target reset button must be pressed, or the board power-cycled.

nrf_test2.c

There are many ways the first test program can be extended, I chose to add serial output (including printf), and also a timeout function based on the ARM systick timer, so the delay function doesn’t hog the CPU. The main loop is:

int main(void)
{
    uint32_t tix;

    mstimeout(&tix, 0);
    init_gpio();
    init_serial();
    printf("\nNRF52 test\n");
    while (1)
    {
        if (mstimeout(&tix, 500))
        {
            NRF_GPIO->OUT ^= LED_BIT;
            putch('.');
        }
        poll_serial();
    }
}

I encountered two obstacles; firstly, I ran out of time trying to understand how to create a non-blocking serial transmit routine using the SDK buffering scheme, so implemented a simple circular buffer that is polled for transmit characters in the main program loop.

The second obstacle was that the CPU systick is a 24-bit down-counter clocked at 64 MHz, which means that it wraps around every 262 milliseconds. So we can’t just use the counter value to check when 500 milliseconds has elapsed, it needs some creative coding to measure that length of time; with hindsight, it might have been better to use a conventional hardware timer.

To build the project just change directory to nrf_test2, and use ‘make’ as before. The source code is fairly self explanatory, but the following features are a bit unusual:

  • For printf serial output, the Arduino programming link on the 6-way connector can’t be used, so we have to select an alternative.
  • A remarkable feature of the UART is that we can choose any unused pin for I/O; the serial signals aren’t tied to specific pins. I’ve arbitrarily chosen I/O pin 15 for output, 14 for input.
  • The method of initialising the UART and the printf output is also somewhat unusual, in that it involves a ‘context’ structure with the overall settings, in addition to the configuration structure.

Viewing serial comms

Serial I/O pins used by nrf_test2
Raspberry Pi SWD and serial connections

The serial output from the target system I/O pin 15 is a 3.3V signal, that is compatible with the serial input pin 10 (BCM 15) on the RPi (TxD -> RxD). To enable this input, launch the Raspberry Pi Configuration utility, select ‘interfaces’, enable the serial port, disable the serial console, and reboot.

To view the serial data, you could install a comms program such as ‘cutecom’, or just enter the following command line in a terminal window (ctrl-C to exit):

stty -F /dev/ttyS0 115200 raw; cat /dev/ttyS0

Debugging

We have already used GDB to program the target system, a similar setup can be used for debugging. Some important points:

  • You’ll be working with 2 binary images; one that is loaded into GDB, and another that has been programmed into the target, and these two images must be identical. If in doubt, you need to reprogram the target.
  • The .elf file that is loaded into GDB contains the binary image and debug symbols, i.e.the names and addresses of your functions & variables. You can load in a .hex file instead, but that has no symbolic information, so debugging will be very difficult.
  • Compiler optimisation is normally enabled (using the -O3 option) as it generates efficient code, but this code is harder to debug, since there isn’t a one-to-one correspondence between a line of source and a block of instructions. Disabling optimisation will make the code larger and slower, but easier to debug; to do this, comment out the OPTIMISE line in the makefile (by placing ‘#’ at the start) and rebuild using ‘make -B’
  • OpenOCD must be running on the Raspberry Pi, configured for SWD mode and the NRF52 processor (files rpi2.cfg and nrf52_swd.cfg). It will be fully remote-controlled from GDB, so won’t require any other files on the RPi.
  • GDB must be invoked in remote mode, with “target remote ADDR:3333” where ADDR is the IP address of the Raspberry Pi, or localhost if GDB and OpenOCD are running on the same machine.
  • GDB commands can be abbreviated providing there is no ambiguity, so ‘print’ can be shortened to ‘p’. Some commands can be repeated by hitting the Enter key, so if the last command was ‘step’, just hit Enter to do another step.

Here is a sample debugging session (user commands in bold):

# On the RPi:
sudo openocd -f ../openocd/rpi2.cfg -f ../openocd/nrf52_swd.cfg -c "bindto 0.0.0.0"

# On the PC, if RPi is at 192.168.1.2:
arm-none-eabi-gdb -ex="target remote 192.168.1.2:3333" build/nrf_test2.elf
Target system halts, current source line is shown

# Program binary image into target system
load
Loading section .text, size 0x215c lma 0x0
Loading section .log_const_data, size 0x10 lma 0x215c
..and so on..

# Print Program Counter (should be at reset handler)
p $pc
$1 = (void (*)()) 0x2b4 <Reset_Handler>

# Execute program (continue)
c

# Halt program: hit ctrl-C, target reports current location
ctrl-C
Program received signal SIGINT, Interrupt.
 main () at nrf_test2.c:72
 72              poll_serial();

# Print millisecond tick count
p msticks
$3 = 78504

# Print O/P port value in hex
p/x NRF_GPIO->OUT
$4 = 0x8080

# Toggle LED pin on O/P port
set NRF_GPIO->OUT ^= 1<<7

# Restart the program from scratch, with breakpoint
set $pc=Reset_Handler
b putch
c
Breakpoint 1, putch (c=13) at nrf_test2.c:149
 149         int in=ser_txin+1;

# Single-step, and print a local variable
s
151         if (in >= SER_TX_BUFFLEN)
p in
$5 = 46

# Detach from remote, and exit
det
quit

Next step

I guess the next step is to get wireless communications working, watch this space…

Copyright (c) Jeremy P Bentham 2019. Please credit iosoft.blog if you use the information or software in here.