RP2040 WiFi using Microchip ATWINC1500 module: part 2

Server sockets

In part 1, we got as far as connecting an ATWINC1500 or 1510 module to a WiFi network; now it is time to do something vaguely useful with it.

Sockets

A network interface is frequently referred to as a ‘socket’ interface, so first I’d better define what that is.

A socket is a logical endpoint for network communication, and consists of an IP address, and a port number. The IP address is often assigned automatically at boot-time, using Dynamic Host Configuration Protocol (DHCP) as in part 1, but can also be pre-programmed into the unit (a ‘static’ address).

The 16-bit port number further subdivides the functionality within that IP address, so one address can support multiple simultaneous conversations (‘connections’). Furthermore, specific port numbers below 1024 are generally associated with specific functions; for example, an un-encrypted Web server normally uses port 80, whilst an encrypted server is on port 443. Port numbers 1024 and above are generally for user programs.

Clients and servers, UDP and TCP

A sever runs all the time, waiting for a client to contact it; the client is responsible for initiating the contact and providing some data, probably in the form of a request. The server returns an appropriate response, then either side may terminate the connection; the client may end it because it has received enough data, or the server because there are limits on the maximum number of simultaneous clients it can service.

There are 2 fundamental communication methods in TCP/IP: User Datagram Protocol (UDP) and Transmission Control Protocol (TCP).

UDP is the simpler of the two, and involves sending a block of data to a given socket (port and IP address) with no guarantee that it will arrive. TCP involves sending a stream of data to a socket; it includes sophisticated retry mechanisms to ensure that the data arrives.

There are those in the networking community who shun UDP, because they think the unreliability makes is useless; I disagree, and think there are various use-cases where the simple block-based transfer is perfectly adequate, possibly overlaid with a simple retry mechanism, so we’ll start with a simple UDP server.

UDP server

The simplest UDP server is stateless, i.e. it doesn’t store any information about the client; it just responds to any request it receives. This means that a single socket can handle multiple clients, unlike TCP which requires a unique socket for each client it is communicating with.

For a classic C socket interface, the steps would be:

  1. Create a datagram socket using socket()
  2. Bind to the socket to a specific port using bind()
  3. When a message is received on that port, get the data, return address and port number using recvfrom()
  4. Send response data to the remote address and port number using sendto()
  5. Go to step 3

The code driving the ATWINC1500 module does the same job, but the function calls are a bit different, as they reflect the messages sent to & received from the WiFi module:

  1. Initialise a socket structure for UDP
  2. Send a BIND command to the module, with the port number
  3. Receive a BIND response
  4. Send a RECVFROM command to the module
  5. Wait until a RECVFROM response is received, get the data, return address & port number
  6. Send a SENDTO command to the module with the response data, return address & port number
  7. Go to step 4

Note that there may be a very long wait between steps 4 and 5, if there are no clients contacting the server. Fortunately the module will signal the arrival of a message by asserting the interrupt request (IRQ) line, so the RP2040 CPU can proceed with other tasks while waiting.

UDP Reception

There are 4 steps when the module receives a packet (‘datagram’) from a remote client:

  1. Get the group ID and operation. This identifies the message type; for UDP it will generally be a response to a RECVFROM request, but it could be something completely different. My software combines the group ID and operation into a single 16-bit number.
  2. Get the operation-specific header. This is generally 16 bytes or less, and in the case of RECVFROM, gives the IP address and port number of the sender, also a pointer & offset to the user data in the buffer.
  3. Get the user data. The application doesn’t need to fetch all the incoming data; for example, in the case of a Web server, it might just get the first line of the page request, and discard all the other information.
  4. Handle socket errors. If there is an error, the data length-value will be negative, and the code must take appropriate action, such as closing and re-opening the server socket. Since a UDP socket is connectionless, it generally won’t see many errors, but a TCP socket will flag an error every time a client closes an active connection.

For RECVFROM, the step 1 & 2 headers are:

// HIF message header
typedef struct {
    uint8_t gid, op;
    uint16_t len;
} HIF_HDR;

// Operation-specific header
typedef struct {
    SOCK_ADDR addr;
    int16_t dlen; // (status)
    uint16_t oset;
    uint8_t sock, x;
    uint16_t session;
} RECV_RESP_MSG;

Having fetched these two blocks, control is passed to a state machine that takes appropriate action. If we’ve just received an indication that DHCP has succeeded, then we bind the server sockets.

    if (gop==GOP_DHCP_CONF)
    {
        for (sock=MIN_SOCKET; sock<MAX_SOCKETS; sock++)
        {
            sp = &sockets[sock];
            if (sp->state==STATE_BINDING)
                put_sock_bind(fd, sock, sp->localport);
        }
    }

When we get a message indicating the binding has succeeded, then if it is a TCP socket, we need to send a LISTEN command. If UDP, we can just send a RECVFROM, and wait for data to arrive. We can tell whether the socket is TCP or UDP by looking at the socket number; the lower numbers are TCP, and higher are UDP.

    else if (gop==GOP_BIND && (sock=rmp->bind.sock)<MAX_SOCKETS &&
             sockets[sock].state==STATE_BINDING)
    {
        sock_state(sock, STATE_BOUND);
        if (sock < MIN_UDP_SOCK)
            put_sock_listen(fd, sock);
        else
            put_sock_recvfrom(fd, sock);
    }

If a UDP server, we may now get a RECVFROM response, indicating that a packet (‘datagram’) has arrived. If so, we save the return socket address (IP and port number), call a handler function, then send another RECVFROM request.

    else if (gop==GOP_RECVFROM && (sock=rmp->recv.sock)<MAX_SOCKETS &&
             (sp=&sockets[sock])->state==STATE_BOUND)
    {
        memcpy(&sp->addr, &rmp->recv.addr, sizeof(SOCK_ADDR));
        if (sp->handler)
            sp->handler(fd, sock, rmp->recv.dlen);
        put_sock_recvfrom(fd, sock);
    }

A very simple handler just echoes back the incoming data:

uint8_t databuff[MAX_DATALEN];

// Handler for UDP echo
void udp_echo_handler(int fd, uint8_t sock, int rxlen)
{
    if (rxlen>0 && get_sock_data(fd, sock, databuff, rxlen))
        put_sock_sendto(fd, sock, databuff, rxlen);
}

UDP client for testing

For testing, I use a simple UDP client written in Python, that can run on a Raspberry Pi, or any PC running Linux or Windows. It sends a message every second, and checks for a response. You’ll need to change the IP address to match the DCHP value given by the module.

# Simple Python client for testing UDP server
import socket, time

ADDR = "10.1.1.11"
PORT = 1025
MESSAGE = b"Test %u"
DELAY = 1

def hex_str(bytes):
    return " ".join([("%02x" % int(b)) for b in bytes])

print("Send to UDP %s:%s" % (ADDR, PORT))
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.settimeout(0.2)
count = 1
while True:
    msg = MESSAGE % count
    sock.sendto(msg, (ADDR, PORT))
    print("Tx %u: %s" % (len(msg), hex_str(msg)))
    count += 1
    try:
        data = sock.recvfrom(1000)
    except:
        data = None
    if data:
        bytes, addr = data
        s = hex_str(bytes)
        print("Rx %u: %s\n" % (len(bytes), s))
    time.sleep(DELAY)

TCP server

A TCP connection is more complex than UDP, since the module firmware must keep track of the data that is sent & received, in order to correct any errors. The steps for a classic C socket interface would be:

  1. Create a ‘stream’ socket using socket()
  2. Bind to the socket to a specific port using bind()
  3. Set the socket to wait for incoming connections using listen()
  4. When a connection request is received on the main socket, open a new socket for the data using accept()
  5. When data arrives on the new socket, get it using recv()
  6. Send response data using send()
  7. If socket error or transfer complete, close socket. Otherwise go to step 5

The corresponding operations for the WiFi module are:

  1. Initialise a socket structure for TCP
  2. Send a BIND command to the module, with the port number
  3. Receive a BIND response
  4. Send a LISTEN command
  5. Receive a LISTEN response
  6. Receive an ACCEPT notification when a connection request arrives on the main socket. Save the new socket number.
  7. Send a RECV command on the new socket.
  8. Receive a RECV response when data arrives on the new socket
  9. Send response data using SEND
  10. Go to step 7, or close the new socket

TCP reception

The first step (binding a socket to a port number) is the same as for UDP, but then we send a LISTEN command, which activates the socket to receive incoming connections. When a client connects, we get an ACCEPT response containing 2 socket numbers; the first is the one that we used for the original BIND command, and the second is a new socket that will be used for the data transfer; we need to issue a RECV on this socket to get the user data.

    else if (gop==GOP_ACCEPT &&
             (sock=rmp->accept.listen_sock)<MAX_SOCKETS &&
             (sock2=rmp->accept.conn_sock)<MAX_SOCKETS &&
             sockets[sock].state==STATE_BOUND)
    {
        memcpy(&sockets[sock2].addr, &rmp->recv.addr, sizeof(SOCK_ADDR));
        sockets[sock2].handler = sockets[sock].handler;
        sock_state(sock2, STATE_CONNECTED);
        put_sock_recv(fd, sock2);
    }

When data is available, the RECV command will return, and we can call a data handler function, then send another RECV for more data. Alternatively, if the data length is negative, then there is an error, and the socket needs to be closed. This isn’t necessarily as bad as it sounds; the most common reason is that the client has closed the connection, and we just need to erase the 2nd socket for future use.

    else if (gop==GOP_RECV && (sock=rmp->recv.sock)<MAX_SOCKETS &&
            (sp=&sockets[sock])->state==STATE_CONNECTED)
    {
        if (sp->handler)
            sp->handler(fd, sock, rmp->recv.dlen);
        if (rmp->recv.dlen > 0)
            put_sock_recv(fd, sock);
    }

The TCP data handler is again a simple echo-back of the incoming data, but with an added complication: if the data length is negative, there has been an error. This isn’t as bad as it sounds; the most common error is that the client has closed the TCP connection, so the server must also close its data socket, to allow it to be re-used for a new connection.

// Handler for TCP echo
void tcp_echo_handler(int fd, uint8_t sock, int rxlen)
{
    if (rxlen < 0)
        put_sock_close(fd, sock);
    else if (rxlen>0 && get_sock_data(fd, sock, databuff, rxlen))
        put_sock_send(fd, sock, databuff, rxlen);
}

TCP client for testing

This is similar to the UDP client; it can run on a Raspberry Pi or PC, running Linux or Windows:

# Simple Python client for testing TCP server
import socket, time

ADDR = "10.1.1.11"
PORT = 1025
MESSAGE = b"Test %u"
DELAY = 1

def hex_str(bytes):
    return " ".join([("%02x" % int(b)) for b in bytes])

print("Send to TCP %s:%s" % (ADDR, PORT))
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(0.5)
sock.connect((ADDR, PORT))
count = 1
while True:
    msg = MESSAGE % count
    sock.sendall(msg)
    print("Tx %u: %s" % (len(msg), hex_str(msg)))
    count += 1
    try:
        data = sock.recv(1000)
    except:
        data = None
    if data:
        s = hex_str(data)
        print("Rx %u: %s\n" % (len(data), s))
    time.sleep(DELAY)
# EOF

Source files

The C source files are in the ‘part2’ directory on  Github here

The default network name and passphrase are “testnet” and “testpass”; these must be changed to match your network, then the code will need to be rebuilt & run using the standard Pico devlopment environment.

The default TCP & UDP port numbers are 1025, and the Python programs I’ve provided can be used to perform simple simple echo tests, providing the IP address is modified to match that given when the Pico joins the network.

python tcp_tx.py
Send to TCP 10.1.1.11:1025
Tx 6: 54 65 73 74 20 31
Rx 6: 54 65 73 74 20 31

Tx 6: 54 65 73 74 20 32
Rx 6: 54 65 73 74 20 32
..and so on..

python udp_tx.py
Send to UDP 10.1.1.11:1025
Tx 6: 54 65 73 74 20 31
Rx 6: 54 65 73 74 20 31

Tx 6: 54 65 73 74 20 32
Rx 6: 54 65 73 74 20 32
..and so on..

Copyright (c) Jeremy P Bentham 2021. Please credit this blog if you use the information or software in it.

PicoReg: real-time diagnostics for the Pi Pico using SWD

PicoReg PyQt display

The Raspberry Pi Pico CPU (RP2040) has a remarkably complex set of peripherals, and this is reflected in the very large number of control registers (1,116).

To debug a C or Python application, it can be very helpful to know the values in these registers; instead of adding ‘print’ calls, or using a heavyweight debugger, you can use a 3-wire connection between the Pico and a Raspberry Pi, and view the state of any register without modifying or disrupting the software. This magic is achieved using the Single Wire Debug (SWD) interface; it is mainly used for reprogramming the Flash memory of the Pico, but can do a lot more – it acts as a transparent window into the I/O subsystems, that is completely independent of the CPU.

This capability can be accessed using software tools such as OpenOCD, GDB, and Eclipse, but I wanted to create something much simpler, and easier to use. The end-result is PicoReg, which is a pure-Python program that runs on any Raspberry Pi. In addition to the SWD interface, it can access the standard System View Description (SVD) file, to give a description of every register in considerable detail.

The end-result is a simple-to-use software tool that gives a valuable insight into the inner workings of the Pico.

Installation

Hardware

Pi -to-Pico SWD connections

Only 3 wires are needed; a ground connection, SWCLK and SWDIO.

Any I/O pins on the Pi could be used; for ease of identification, I’ve chosen BCM pin numbers 20 and 21. These should be connected to the SWCLK and SWDIO pins on the edge of the Pico; keep the wires as short as possible, ideally a 150 mm (6 inches) or less.

The pins are defined at the top of picoreg_gpio.py, so can easily be changed to any others, but you need to be sure there is no conflict with other device drivers, such as serial, SPI etc.

# Clock and data BCM pin numbers
CLK_PIN         = 20
DAT_PIN         = 21

The Pico will need to be powered as normal, for example using a 5V supply into the USB socket, or if the Pico USB interface is not connected, linking the the 5V pin on the Pi to the VSYS pin on the Pico.

Software

There are 2 Python files, and one database file:

  • picoreg_gpio.py. The low-level interface code, with a very simple command-line interface.
  • picoreg_qt.py. A PyQt application with full GUI, that uses picoreg_gpio as a low-level interface.
  • rp2040.svd. The System View Description file provided by the Raspberry Pi organisation, that describes all the peripheral registers in XML format

The GUI translates register names (such as TIMER.ALARM3) into physical addresses (such as 0x4005401c) using the SVD file; if it is missing, the GUI won’t work. The software performs best on a Pi 3 or 4; it will work on earlier devices, but is a bit slower.

The files can be found on github here; copy them to any convenient directory, then install PyQt5 on the Pi. The current version at the time of writing is 5.11.3:

sudo apt update
sudo apt install python3-pyqt5

That is all you need to do!

Running the GUI

Start the GUI from a console:

python3 picoreg_qt.py

The application will respond “Loading rp2040.vsd”, then after a short delay, will show the GUI. You may also see some warning messages from PyQt, but these are harmless.

Picoreg initial display

The controls are:

  • Core number. The Pico is a 2-core device, and this allows you to select core 0 or 1. There is no practical difference when using Picoreg to look at I/O, since the peripheral registers are common to both cores.
  • Verbose. This allows you to see the SWD messages that Picoreg is sending, and the responses obtained; useful when there are problems with the link.
  • Connect. This starts communication with the Pico, and verifies the identity of the RP2040 CPU. When the link is broken, it is necessary to re-connect before doing any register accesses.
  • Single. When connected, this button takes a single reading from the highlighted register.
  • Run. When connected, this button starts a continuous cycle of reading from the highlighted register, at 5 readings per second.

Note that this initial display is not Raspberry-Pi-specific; it will run on any PC, even under Windows. This allows you to browse the register database on any convenient machine, though the low-level I/O code is Pi-specific (using the RPI.GPIO module), so the SWD code only runs on a Pi.

The upper display is a tree structure, containing the peripherals, registers, and fields within the registers. By default it is alphabetically sorted, click on the ‘base’ header, to sort by address. The lower display shows general information, and a description of the selected register.

If the SVD database file is damaged or missing, Picoreg will report a syntax error; check that the file is in the same directory as picroreg_qy.py, and restart.

Click on ‘connect’ and if all is well, the following will be displayed.

SWD connection restart
DPIDR 0x0bc12477

The Debug Port Identification Register value shows that Picoreg connection has succeeded. The software makes 3 attempts to do this, and if unsuccessful, the most likely cause is incorrect wiring, or the Pico board not being powered up.

You can navigate around the register display using mouse (single or double-click) or keyboard, as is usual for such tree displays.

If you have a new un-programmed Pico, try navigating to the TIMER.TIMELR register, and hit ‘Run’.

Viewing timer value

You should see a rapidly-changing display of the current timer value; you can then move the cursor to other registers, whilst the data collection is still running; the register value under the cursor will be updated, while previously-accessed register values are static. There is a flashing indication in the bottom-left corner of the window to show that data collection is running.

Debugging an application

Load the following MicroPython application onto the Pico

# Simple test of O/P and ADC
from machine import Pin, Timer, ADC
led = Pin(25, Pin.OUT)
temperature = ADC(4)
timer = Timer()

def blink(timer):
    led.toggle()
    print(temperature.read_u16())

timer.init(freq=2, mode=Timer.PERIODIC, callback=blink)

The on-board LED will flash at 1 Hz, and the console will report the ADC value on the temperature channel.

You can use PicoReg to display the raw ADC value, in the ADC.RESULT register:

The state of the I/O pins is in the SIO.SPIO_IN register:


You can even see activity on the USB link, as the data pins toggle high and low:

Potential issues

Styling

This proved to be a surprising headache; it has been remarkably difficult to get consistent styling. Some of the screenshots were taken on a Pi 3 running Qt 5.11.3, remote-controlled from a PC, using SSH:

export DISPLAY=:0.0
python3 picoreg_qt.py

The others are run directly from a Pi console, and look quite different (and not as nice, in my opinion).

I’ve already used some very limited styling to customise the tree display:

TREE_STYLE    = ("QTreeView{selection-color:#FF0000;} " +
                 "QTreeView{selection-background-color:#FFEEEE;} ")

self.tree.setStyleSheet(TREE_STYLE)

However, after many hours of experimentation, I still haven’t found a way of getting a consistent appearance – feel free to tackle this problem yourself!

Errors

PicoReg uses pure-python code; on the plus side, you get the convenience of not having to install any device-drivers, but on the minus-side the SWD timing is a bit unpredictable, and can be stretched out when a higher-priority task takes control of the CPU. This is particularly noticeable when resizing or repositioning the display window; the software makes 3 attempts to re-establish connection with the Pico, but may time out and disconnect.

This behaviour is harmless, and it is only necessary to click the ‘Connect’ button to re-establish communications.

Console interface

If there are problems running the graphical interface, the low-level drivers in can be run directly from the command line:

python3 picoreg_gpio.py [options] [address]
    options: -v Verbose mode
             -r Repeated access 
    address: hexadecimal address to be monitored

The default address to be monitored is the GPIO input 0xD0000004. The low-level driver can’t access the SVD database, so the address has to be specified in hexadecimal, e.g. to display the timer value:

python3 picoreg_gpio.py -r 0x4005400c
SWD connection restart
DPIDR 0x0bc12477
0x4005400c: 0xefa1b0d0
0x4005400c: 0xefa26046
0x4005400c: 0xefa30fc8
0x4005400c: 0xefa3bf32
0x4005400c: 0xefa46ec3
..and so on, use ctrl-C to exit

How it works

The SWD link has 2 wires: a clock line, and a bi-directional data line. All transactions are initiated by the Pi, the Pico only transmits on the data line when requested. The data is sent LSB (Least Significant Bit) first.

Each message starts with an 8-bit header, and the Pico responds with a 3-bit acknowledgement; if this is OK (bit value 1, 0, 0) then a 32-bit data value is transferred (sent or received), followed by a parity bit.

As there is only one data line, the Pi has to switch its direction from transmit to receive to get the acknowledgement, and then switch back to transmit (if it is a write cycle) or allow the Pico to send 32 bits of data (if it is a read cycle).

An extra complication that isn’t emphasised in the ARM documentation, is that the active edge changes as well; the Pi sends data that is stable on the rising clock edge, but the Pico data is stable on the falling clock edge, as shown in the following oscilloscope trace. [The response from the Pico has been amplified for clarity; normally it is the same magnitude as the outgoing signal from the Pi.]

SWD header transfer; blue trace is clock (1 MHz), red trace is data

The unusual elements of this protocol make it quite tricky to implement in pure Python, and I must admit the above trace wasn’t generated by my code; it comes from OpenOCD, with an FTDI USB adaptor. The following trace is generated by my code, the frequency is a bit lower (approximately 100 kHz) and is less symmetrical.

SWD header, using Python GPIO

The asymmetric waveform isn’t a problem, but a disadvantage of the pure-Python approach is that there are occasions when the CPU is performing other tasks, and it stop driving the SWD interface.

The trace below shows a 300 microsecond pause in the middle of a transfer, and unsurprisingly the Pico doesn’t like this, and returns an error response. Occasional errors like this aren’t a problem, as they are easily handled by the retry mechanism in the PicoReg code.

SWD transfer with gap in transmission

Error handling

When there is an error, the Pico completely stops communicating over the SWD interface, and ignores all subsequent commands; it is necessary to reset the SWD interface, to re-establish communication.

The reset process is deliberately quite complex, involving the transmission of:

  • 8 all-1 bits (FF hex)
  • 16 bytes of a specific polynomial
  • 4 all-0 bits (0 hex)
  • 1 byte to activate the SW-DP interface
  • At least 50 all-1 bits

If there are any errors in the process, the Pico will not respond, which makes debugging a bit tricky. Once reset, the connection has to re-established by, transmitting the ID of the core to be accessed; the RP2040 processor has two cores, that are selected using a value of 0x01002927 or 0x11002927. When browsing the peripheral registers, it doesn’t matter which core is selected; the results will be the same.

Higher-level protocol

You might think that, having established contact with the CPU, it would be easy to read the register values, but in reality the interface has several extra levels of complication; some of these I’ve already documented in a previous project, but if you are seeking more information, you really need to read the Arm Debug Interface Architecture Specification (ADI). At the time of writing the latest version (v6.0) is available here.

Copyright (c) Jeremy P Bentham 2021. Please credit this blog if you use the information or software in it.