3D design with Python and FreeCAD

3D CAD packages can be hard work; there is a lot to learn, which can be a major problem for an infrequent user such as myself.

Most packages support some form of scripting, so why not program my complete design from scratch, without touching the GUI? FreeCAD is a (free) 3D design package, with a comprehensive Python interface, so seems to be ideal…

freecad5

This is simple in theory, but a bit tricky in practice; I’ll spare you the many frustrating false-starts I’ve made, and describe some simple ways of producing 3D objects from scratch in Python. This is very much a work-in-progress, but hopefully will provide some useful pointers if you’re a Python programmer doing occasional 3D design.

The examples here have been tested with FreeCAD v0.16, and the current version 0.18

Running Python code

There are several ways of running a Python script in FreeCAD:

1. Entering commands at the console

In the FreeCAD Python console window, try entering:

FreeCAD.newDocument("Unnamed")
import Part
box = Part.makeBox(4, 3, 2)
Part.show(box)

The result is a bit underwhelming; all you can see is the bottom left-hand corner of a square. If you want to see the box in its full 3-D glory, either use the GUI controls to change the viewpoint, or add the following 2 lines:

FreeCADGui.activeDocument().activeView().viewAxonometric()
FreeCADGui.SendMsgToActiveView("ViewFit")
2. Executing as a Macro

Click on Macro then Macros… and you are given a list of Python macro files that can be executed. They are stored in the default location for scripts; you can alter this to a directory of your choosing, by changing the ‘user macros location’ in the dialog box. If you make that change, it is necessary to exit & re-enter FreeCAD for the change to take effect.

If FreeCAD encounters a problem with your script, it will generally give a sensible error message, however very occasionally a script can corrupt the internals of the program, so it fails to respond in the usual manner. Hence, if you are experiencing problems with previously-good code that suddenly doesn’t work, it is worth restarting FreeCAD in case this fixes the problem.

3. Using the FreeCAD editor

You can load a Python script into FreeCAD using the normal GUI File Open. You are then presented with a nice-looking editor window, into which you can paste one of the examples from this blog. When complete, the file can be run by pressing ctrl-F6. Unfortunately, there are some subtle differences when executing a file in this manner, as opposed to the other methods, see my usage of the recompute() function in the later code examples.

4. Executing a file from the FreeCAD Python console

You can directly execute a file by entering a command at the Python console, e.g.

exec(open("/Projects/FreeCAD/test.py").read())

 

Design methodology

Like many 3D CAD packages, FreeCAD uses the Constructive Solid Geometry (CSG) method, where the final design is built up by adding (fusing) elements together, and subtracting (cutting) one element from another. Simple 3D  objects (cube, cylinder etc.) can be created with a single line of code:

# Simple test of FreeCAD Part scripting, from iosoft.blog

from FreeCAD import Vector
import Part

if FreeCAD.ActiveDocument:
    FreeCAD.closeDocument("Unnamed")
doc = FreeCAD.newDocument("Unnamed")
plate = Part.makeBox(50, 30, 2)
verticals = [edge for edge in plate.Edges if edge.BoundBox.ZLength]
plate = plate.makeFillet(5, verticals)
cyl = Part.makeCylinder(10, 2, Vector(20, 15, 0))
plate = plate.cut(cyl)
doc.addObject("Part::Feature", "plate").Shape = plate
FreeCADGui.activeDocument().activeView().viewAxonometric()
FreeCADGui.SendMsgToActiveView("ViewFit")

The result is a plate with rounded corners, and a large off-centre hole:

freecad1

However the code is more than a single line, so some explanations are in order:

Lines 6-8 remove a previous unnamed document, and create a new one. This means that every time you run the script you get a new clean document to display the result. As a safeguard, if you currently have a named document open, the script will error out

Line 9 creates a square plate, size 50 x 30 units, and 2 units thick.

Lines 10 & 11 create the rounded corners (‘fillets’ in FreeCAD terminology), with a radius of 5 units. The makeFillet method requires a list of edges, and we only want to modify the vertical edges, so the list of edges is filtered by checking the z-dimension length is non-zero.

Lines 12 & 13 create a cylinder with 10 units radius and 2 units high, offset from the origin using a ‘Vector’ object, which defines a position in 3-dimensional (x,y,z) space. The plate is then cut with the cylinder, creating a hole 20 units in diameter.

Line 14 adds the resulting object to the current document; without this step, the object won’t be visible.

Lines 15 & 16 set a perspective view, and adjust the zoom level so the part fits in the display space.

The ‘labels and attributes’ on the left-hand side of the screen show there is only one defined object, named ‘plate’; if you click on that you can modify its placement (i.e. its position and orientation) but none of the other design parameters.

Workbenches

FreeCAD has various sets of software tools, divided up into ‘workbenches’. This division isn’t just for the GUI, it also applies to scripting; for example, the above Python code used tools imported from the Part workbench. There is some overlap between workbenches, so there can be other ways of creating the same object, for example starting with 2-dimensional sketches in the Draft workbench:


# Simple test of FreeCAD Draft scripting, from iosoft.blog

from FreeCAD import Vector
import Draft

if FreeCAD.ActiveDocument:
    FreeCAD.closeDocument("Unnamed")
doc = FreeCAD.newDocument("Unnamed")

rect = Draft.makeRectangle(50, 30, face=True)
rect.FilletRadius = 5
rect.ViewObject.Visibility = False
plate = doc.addObject("Part::Extrusion", "plate")
plate.Base, plate.Dir = rect, Vector(0, 0, 2)

circ = Draft.makeCircle(10, face=True)
rotation = App.Rotation(Vector(0,0,0), 0)
circ.Placement = App.Placement(Vector(20, 15, 0), rotation)
circ.ViewObject.Visibility = False
cyl = doc.addObject("Part::Extrusion", "cylinder")
cyl.Base, cyl.Dir = circ, Vector(0, 0, 2)

FreeCADGui.activeDocument().activeView().viewAxonometric()
FreeCADGui.SendMsgToActiveView("ViewFit")

cutplate = doc.addObject("Part::Cut", "cutplate")
cutplate.Base, cutplate.Tool = plate, cyl

doc.recompute()

To turn the 2-dimensional x-y sketches into 3-dimensional objects, they are extruded in the z-plane. The Draft workbench has no extrusion capability, so this is done by adding extrusion objects to the document. It is important to set the original 2-dimensional sketches as transparent (set visibility false), otherwise they will form a thin layer that obscures the cutout.

I must explain the last 5 lines of code, where I set the viewing mode, cut the plate with the cylinder, then do a recompute. This is a workaround for some minor issues I found in the current FreeCAD versions, which may well be fixed by now:

  • If the view mode is set after doing the cut, you don’t get the correct perspective view.
  • If the final recompute is omitted, everything works fine when executed as a standalone script, but when the same script is executed from the edit screen, the document is blank; none of the objects are visible.

Apart from these issues, the graphical end result looks exactly the same as with the Part workbench, but note the collection of nested objects in the left-hand window. This is CSG in action; the cut plate is derived from a plate and a cylinder, which in turn are derived from a rectangle and circle.

freecad2

An advantage of this object hierarchy is that the design parameters can be changed within the GUI; for example, try changing the circle radius in the attributes window from 10 to 14 units, hit F5 to recompute, and the hole diameter will increase to 28 units. The primary focus of this blog is to use scripting rather than the GUI, but it can be useful to manually change a few parameters, and instantly see the result.

An interesting quirk of this hierarchy is that the cut plate has effectively taken possession of the circle and cylinder, since they are its sub-objects. This raises the question: what happens if we cut 2 plates with the same cylinder, which of the plates will own the cylinder? Let’s try it out:


# Simple test of FreeCAD CSG hierarchy, from iosoft.blog

from FreeCAD import Vector
import Draft

if FreeCAD.ActiveDocument:
    FreeCAD.closeDocument("Unnamed")
doc = FreeCAD.newDocument("Unnamed")

rect = Draft.makeRectangle(50, 30, face=True)
rect.FilletRadius = 5
rect.ViewObject.Visibility = False
plate = doc.addObject("Part::Extrusion", "plate")
plate.Base, plate.Dir = rect, Vector(0, 0, 2)

plate2 = doc.addObject("Part::Extrusion", "plate2")
plate2.Base, plate2.Dir = rect, Vector(0, 0, 2)
rotation = App.Rotation(Vector(0,0,0), 0)
plate2.Placement = App.Placement(Vector(0, 0, 10), rotation)

circ = Draft.makeCircle(10, face=True)
circ.Placement = App.Placement(Vector(20, 15, 0), rotation)
circ.ViewObject.Visibility = False
cyl = doc.addObject("Part::Extrusion", "cylinder")
cyl.Base, cyl.Dir = circ, Vector(0, 0, 12)

FreeCADGui.activeDocument().activeView().viewAxonometric()
FreeCADGui.SendMsgToActiveView("ViewFit")

cutplate = doc.addObject("Part::Cut", "cutplate")
cutplate.Base, cutplate.Tool = plate, cyl

cutplate2 = doc.addObject("Part::Cut", "cutplate2")
cutplate2.Base, cutplate2.Tool = plate2, cyl

doc.recompute()

..and the answer to the question is..

freecad3

FreeCAD v0.18 has duplicated the single cylinder, so it appears twice in the hierarchy. Even though they appear to be separate, these 2 circles & cylinders are actually linked; if you change the radius of one circle, both hole sizes change. The trick is to note that the duplicate objects have the same name; if they were separate items, FreeCAD would have automatically renamed the second one by adding a numeric suffix.

FreeCAD v0.16 does not duplicate the parts, the circle & cylinder only appear once under cutplate2, so caution is needed when working with the older version.

Copyright (c) Jeremy P Bentham 2019. Please credit this blog if you use the information or software in it.

PyQt serial terminal

A simple demonstration of threading in PyQt

I do a lot of work with serial comms; TeraTerm is an excellent serial terminal, but sometimes a customised application is required, for example when dealing with binary data.

The following blog describes a simple Windows & Linux serial terminal, that can be adapted to handle special protocols; it also demonstrates the creation of Python threads, and can serve as a basis for other multi-threaded applications.

pyqt_serialterm1

It won’t win any awards for style, but could come in handy next time you encounter an obscure serial protocol.

Compatibility

My code is compatible with Python 2.7 and 3.x, PyQt v4 and v5, running on Windows or Linux. This necessitates some rather clunky inclusions at the top of the file:

try:
    from PyQt4 import QtGui, QtCore
    from PyQt4.QtGui import QTextEdit, QWidget, QApplication, QVBoxLayout
except:
    from PyQt5 import QtGui, QtCore
    from PyQt5.QtWidgets import QTextEdit, QWidget, QApplication, QVBoxLayout
try:
    import Queue
except:
    import queue as Queue

There is also an issue with the different ways Python 2 and 3 handle serial data; older versions assume that the data type is an ASCII string, while in later versions it is a binary type.

This can lead to all sorts of problems (such as unwanted exceptions) so I’ve included specific functions to ensure that all outgoing data is converted from the internal representation (which could be Unicode) to a string of bytes, and incoming data is converted from the external type (data bytes or string) to a string type:

# Convert a string to bytes (for Python 3)
def str_bytes(s):
    return s.encode('latin-1')

# Convert bytes to string (if Python 3)
def bytes_str(d):
    return d if type(d) is str else "".join([chr(b) for b in d])

You’ll need to install pySerial by the usual methods (e.g. ‘pip install’).

Using threading

The key problem with serial communications is the time it takes; the slower the data rate, the longer the delay before anything happens. Without threading (or any other mitigation scheme) the User Interface (UI) will lock up after each command, waiting for the response. At best, this makes the application appear sluggish and unresponsive; at worst, it can appear to have failed, waiting for a response that never comes.

Threading allows the UI to carry on interacting with the user, while simultaneously keeping the serial link alive. Creating a new thread is really easy in Python; you just subclass QThread, and instantiate it:

class SerialThread(QtCore.QThread):
    def __init__(self, portname, baudrate):
        QtCore.QThread.__init__(self)
        self.portname, self.baudrate = portname, baudrate
...
class MyWidget(QWidget):
    ...
    self.serth = SerialThread(portname, baudrate)

I have chosen to supply the serial parameters (port name & baud rate) when the thread object is instantiated, so they are available when the thread is started; this is done by calling the start() method, which will call a run() method in the QThread class:

# In serial thread:
def run(self):
    print("Opening %s at %u baud" % (self.portname, self.baudrate))
...
# In UI thread:
    self.serth.start()

The run() method needs to keep running in a perpetual loop, but it must be possible to terminate that loop when the program exits – Python won’t do it automatically. I use a global variable, that can be set false to terminate, e.g. in pseudocode:

def run(self):
    [starting: open serial port]
    while self.running:
        [check for incoming characters]
        [check for outgoing characters]
    [finished: close serial port]

When the application is closing, it terminates the thread by setting the boolean variable false, then (very importantly) waits for the thread to finish its execution:

def closeEvent(self, event):
    self.serth.running = False
    self.serth.wait()

Transmitting

The user will be entering keystrokes in the UI thread, and it is tempting to call the serial transmit function from that thread, but this isn’t a good idea; it is better to pass the keystrokes across to the serial thread for transmission, and we need a thread-safe method of doing this. That means we can’t just use a global shared string variable; Python does a lot of behind-the-scenes processing that could lead to an unpredictable result. Instead, we’ll use a First In First Out (FIFO) queue:

# In UI thread..
txq = Queue.Queue()
...
txq.put(s)           # Add string to queue
...
# In serial thread..
if not txq.empty():
    txd = txq.get()  # Get string from queue

So the serial thread polls the transmit queue for any data, outputting it to the serial port.

Receiving

We could use the same technique for received data; the serial thread could add it to a queue that is polled by the UI thread, and somehow trigger a UI redraw when the new data arrives, but I prefer to use a signal; the data is attached to that signal, and is received by a UI function that has registered a connection. The signal has to be in a class definition, and must specify the type of data that will be attached:

class MyWidget(QWidget):
    text_update = QtCore.pyqtSignal(str)

The signal is connected to a function that will process the data:

self.text_update.connect(self.append_text)

So now the serial thread just has to generate a signal when new data is received:

self.text_update.emit(text)

This technique would be quite adequate, but I do like having the output from all my ‘print’ function calls redirected to the same window; it makes for cleaner error reporting when things go wrong, rather than having a separate console with error messages. This is done by redirecting stdout to my widget, and adding write() and flush() handlers:

class MyWidget(QWidget):
    text_update = QtCore.pyqtSignal(str)
    def __init__(self, *args): 
        ...
        self.text_update.connect(self.append_text)
        sys.stdout = self
        ...
    def write(self, text):
        self.text_update.emit(text)  
    def flush(self):
        pass
        ...
    def append_text(self, text):        
        [add text to UI display]

So now, every time I make a print() call, a signal is sent to my append_text function, where the display is updated. The use of a signal means that I can still call print() from any thread, without fear of strange cross-threading problems.

Polling in serial thread

The serial thread is just polling for incoming and outgoing characters, and if there are none, the processor will execute the ‘while’ loop really quickly. In the absence of any delays, it will consume a lot of CPU time just checking for things that don’t exist. This may appear harmless, but it is quite alarming for the user if the CPU fan starts spinning rapidly whenever your application is running, and a laptop user won’t be happy if you needlessly drain their battery by performing pointless tasks. So we need to add some harmless time-wasting to the polling loop, by frequently returning control to the operating system. This can be done by calling the ‘sleep’ function, but we still want the software to be responsive when some serial data actually arrives. A suitable compromise is to use a serial read-function with a timeout, so the software ‘blocks’ (i.e. stalls) until either some characters are received, or there is a timeout:

self.ser = serial.Serial(self.portname, self.baudrate, timeout=SER_TIMEOUT)
...
s = self.ser.read(self.ser.in_waiting or 1)

In case you are unfamiliar with the usage, the ‘or’ function returns the left-hand side if it is true (non-zero), otherwise the right-hand side. So every read attempt is at least 1 character, or more characters if they are available. If none are present, the read function waits until the timeout, so when the serial line is idle, most time will be spent waiting in the operating system.

User Interface

This has been kept as simple as possible, with just a text box as a main widget. One minor complication is that as standard, the text box (which is actually a QTextEdit control) will capture and display all keystrokes, so we need to subclass it to intercept the keys, and call a handler function that adds them to the serial transmit queue. I didn’t want to burden the text box with this functionality, so put the handler in its parent, which is the main widget:

class MyTextBox(QTextEdit):
    def __init__(self, *args): 
        QTextEdit.__init__(self, *args)

    def keyPressEvent(self, event):
        self.parent().keypress_handler(event)

The keystroke handler in the main widget gets the character from the key event, and checks for a ctrl-V ‘paste’ request; I’ve included this feature because I find it useful to cut-and-paste frequently-used serial commands from a document, rather than re-typing them every time.

# In main widget:
def keypress_handler(self, event):
    k = event.key()
    s = RETURN_CHAR if k==QtCore.Qt.Key_Return else event.text()
    if len(s)>0 and s[0]==PASTE_CHAR:
        cb = QApplication.clipboard() 
        self.serth.ser_out(cb.text())
    else:
        self.serth.ser_out(s)

Interestingly, with PyQt5 you can cut-and-paste full 8-bit data (i.e. bytes with the high bit set), but this doesn’t seem to work in PyQt4, which only accepts the usual ASCII character set.

I haven’t included any menu options, you have to specify the COM port on the command-line using the -c option, and baud rate using -b. There is also a -x option to display incoming data in hexadecimal, for example:

Windows:
  python pyqt_serialterm.py -c com2 -b 9600 -x
Linux:
  python pyqt_serialterm.py -c /dev/ttyUSB0 -b 115200

On Linux, if you want non-root access to a serial port, you will generally need to add your username to the ‘dialout’ group:

sudo usermod -a -G dialout $USER

..then log out & back in.

Source code

# PyQt serial terminal from iosoft.blog

VERSION = "v0.09"

try:
    from PyQt4 import QtGui, QtCore
    from PyQt4.QtGui import QTextEdit, QWidget, QApplication, QVBoxLayout
except:
    from PyQt5 import QtGui, QtCore
    from PyQt5.QtWidgets import QTextEdit, QWidget, QApplication, QVBoxLayout
try:
    import Queue
except:
    import queue as Queue
import sys, time, serial

WIN_WIDTH, WIN_HEIGHT = 684, 400    # Window size
SER_TIMEOUT = 0.1                   # Timeout for serial Rx
RETURN_CHAR = "\n"                  # Char to be sent when Enter key pressed
PASTE_CHAR  = "\x16"                # Ctrl code for clipboard paste
baudrate    = 115200                # Default baud rate
portname    = "COM1"                # Default port name
hexmode     = False                 # Flag to enable hex display

# Convert a string to bytes
def str_bytes(s):
    return s.encode('latin-1')

# Convert bytes to string
def bytes_str(d):
    return d if type(d) is str else "".join([chr(b) for b in d])

# Return hexadecimal values of data
def hexdump(data):
    return " ".join(["%02X" % ord(b) for b in data])

# Return a string with high-bit chars replaced by hex values
def textdump(data):
    return "".join(["[%02X]" % ord(b) if b>'\x7e' else b for b in data])

# Display incoming serial data
def display(s):
    if not hexmode:
        sys.stdout.write(textdump(str(s)))
    else:
        sys.stdout.write(hexdump(s) + ' ')

# Custom text box, catching keystrokes
class MyTextBox(QTextEdit):
    def __init__(self, *args):
        QTextEdit.__init__(self, *args)

    def keyPressEvent(self, event):     # Send keypress to parent's handler
        self.parent().keypress_handler(event)

# Main widget
class MyWidget(QWidget):
    text_update = QtCore.pyqtSignal(str)

    def __init__(self, *args):
        QWidget.__init__(self, *args)
        self.textbox = MyTextBox()              # Create custom text box
        font = QtGui.QFont()
        font.setFamily("Courier New")           # Monospaced font
        font.setPointSize(10)
        self.textbox.setFont(font)
        layout = QVBoxLayout()
        layout.addWidget(self.textbox)
        self.setLayout(layout)
        self.resize(WIN_WIDTH, WIN_HEIGHT)      # Set window size
        self.text_update.connect(self.append_text)      # Connect text update to handler
        sys.stdout = self                               # Redirect sys.stdout to self
        self.serth = SerialThread(portname, baudrate)   # Start serial thread
        self.serth.start()

    def write(self, text):                      # Handle sys.stdout.write: update display
        self.text_update.emit(text)             # Send signal to synchronise call with main thread

    def flush(self):                            # Handle sys.stdout.flush: do nothing
        pass

    def append_text(self, text):                # Text display update handler
        cur = self.textbox.textCursor()
        cur.movePosition(QtGui.QTextCursor.End) # Move cursor to end of text
        s = str(text)
        while s:
            head,sep,s = s.partition("\n")      # Split line at LF
            cur.insertText(head)                # Insert text at cursor
            if sep:                             # New line if LF
                cur.insertBlock()
        self.textbox.setTextCursor(cur)         # Update visible cursor

    def keypress_handler(self, event):          # Handle keypress from text box
        k = event.key()
        s = RETURN_CHAR if k==QtCore.Qt.Key_Return else event.text()
        if len(s)>0 and s[0]==PASTE_CHAR:       # Detect ctrl-V paste
            cb = QApplication.clipboard()
            self.serth.ser_out(cb.text())       # Send paste string to serial driver
        else:
            self.serth.ser_out(s)               # ..or send keystroke

    def closeEvent(self, event):                # Window closing
        self.serth.running = False              # Wait until serial thread terminates
        self.serth.wait()

# Thread to handle incoming & outgoing serial data
class SerialThread(QtCore.QThread):
    def __init__(self, portname, baudrate): # Initialise with serial port details
        QtCore.QThread.__init__(self)
        self.portname, self.baudrate = portname, baudrate
        self.txq = Queue.Queue()
        self.running = True

    def ser_out(self, s):                   # Write outgoing data to serial port if open
        self.txq.put(s)                     # ..using a queue to sync with reader thread

    def ser_in(self, s):                    # Write incoming serial data to screen
        display(s)

    def run(self):                          # Run serial reader thread
        print("Opening %s at %u baud %s" % (self.portname, self.baudrate,
              "(hex display)" if hexmode else ""))
        try:
            self.ser = serial.Serial(self.portname, self.baudrate, timeout=SER_TIMEOUT)
            time.sleep(SER_TIMEOUT*1.2)
            self.ser.flushInput()
        except:
            self.ser = None
        if not self.ser:
            print("Can't open port")
            self.running = False
        while self.running:
            s = self.ser.read(self.ser.in_waiting or 1)
            if s:                                       # Get data from serial port
                self.ser_in(bytes_str(s))               # ..and convert to string
            if not self.txq.empty():
                txd = str(self.txq.get())               # If Tx data in queue, write to serial port
                self.ser.write(str_bytes(txd))
        if self.ser:                                    # Close serial port when thread finished
            self.ser.close()
            self.ser = None

if __name__ == "__main__":
    app = QApplication(sys.argv)
    opt = err = None
    for arg in sys.argv[1:]:                # Process command-line options
        if len(arg)==2 and arg[0]=="-":
            opt = arg.lower()
            if opt == '-x':                 # -X: display incoming data in hex
                hexmode = True
                opt = None
        else:
            if opt == '-b':                 # -B num: baud rate, e.g. '9600'
                try:
                    baudrate = int(arg)
                except:
                    err = "Invalid baudrate '%s'" % arg
            elif opt == '-c':               # -C port: serial port name, e.g. 'COM1'
                portname = arg
    if err:
        print(err)
        sys.exit(1)
    w = MyWidget()
    w.setWindowTitle('PyQT Serial Terminal ' + VERSION)
    w.show()
    sys.exit(app.exec_())

# EOF

 

Copyright (c) Jeremy P Bentham 2019. Please credit this blog if you use the information or software in it.

Programming PSoC: an ARM CPU with programmable hardware

Want to craft your own CPU peripheral? Want to experiment with programmable hardware, but are deterred by the complexity and cost? Read on…

Exif_JPEG_PICTURE
Cypress PSoC kit

The Cypress PSoC family are strange devices; a conventional CPU combined with programmable hardware, that gives unparalleled flexibility to any embedded system.

However, this flexibility means that the ‘PSoC Creator’ development environment is unlike anything else, and can be a bit intimidating for any developer who isn’t used to programmable hardware.

In this blog, I try to de-mystify the development process, hopefully demonstrating that the learning curve isn’t as bad as you might think; you can create a simple demonstration from scratch quicker than on most other platforms.

Why use a PSoC?

  • Combines a conventional CPU with programmable logic
  • Ideal for learning about programmable hardware
  • Completely free development environment – no registration required
  • Low-cost development boards (under US $15)
  • Powerful logic optimisation software
  • Comprehensive analogue capability
  • Vast amount of online documentation and worked examples
  • Wide supply-voltage range (1.71 to 5.5 volts).
  • USB connectivity, simple Bluetooth add-on

So why isn’t everyone using these devices? Well, I must admit that when I first ran their ‘PSoC Creator’ IDE, it was a bit overwhelming – there are so many features it is difficult to know where to start, and I’m too impatient to sit through all the training videos.

So I’ll try to get you up to speed quickly, on the assumption you are familiar with the usual process for developing embedded software. The project I’ve chosen is really trivial – flashing an LED – but this is always the first step I take with any new hardware, and in this case I’ll start by using conventional software techniques, then show how the same application can be implemented purely in hardware, with the CPU doing nothing!

.. and in case you’re wondering whether I’m being paid by Cypress to create an overly-enthusiastic write-up, the answer is no, I have no commercial relationship with them: I just think this is really interesting technology, that deserves a wider audience.

Hardware

Exif_JPEG_PICTURE

For this demonstration, all you need is the Cypress CY8CKIT-059, which is a self-contained evaluation board with a detachable USB interface for programming and debugging. It has a PSoC 5LP device with:

  • 80 MHz ARM Cortex-M3 CPU
  • 256 kbytes flash, 64 kbytes RAM
  • 38 GPIO pins
  • Full-speed USB
  • 24 ‘Programmable Universal Digital Blocks’ (known as UDBs)

The board is in a conventional dual-in-line format, ideal for prototyping on 0.1″ pitch breadboard, and at under US $15, you’re getting a lot of hardware for the money.

The board has 2 snap-apart sections, a ‘kitprog’ programmer on the left, and the actual PSoc on the right. For the time being, I’d suggest you keep the board intact, as you lose some handy features (e.g. a debug serial link to the PC) if the sections are snapped apart and re-joined by an SWD cable.

PSoc Creator

psoc_creator
PSoC Creator

This is a Windows-only application; I’m using version 4.2, and (unlike most programmable logic software) you don’t have to register, or get license keys from the vendor – all the features are available for free, right from the start.

I’ll admit that it can be a bit intimidating to a software developer; the central section in the image above looks like (and is) a circuit schematic, and you don’t normally get that in a software IDE. Once you start using the software, it gets even stranger, as every time you build the application, umpteen C and Verilog files are created; you can get an out-of-control feeling when faced with all the automatically-generated files you didn’t write.

However, fairly quickly you get to appreciate the amount of work that has gone into these pre-prepared (and very well-documented) hardware & software devices – they do make your job a lot easier, and having access to the source files is excellent for experimentation & learning.

Flashing LED example

We’ll start by creating a conventional flashing-LED example in software, then transition to a programmable-hardware version.

When PSOC Creator is launched, you’ll see a Start Page, with the usual option of creating a new Design Project. Click this, and select the Target Device as PSoC 5LP Cy8C5888LTI-LP097. If you get this wrong, no great harm will result, the device programming will fail, with a helpful message as to the device you’ve set, and the device that is actually fitted to the board.

You’ll then be asked whether you want to run a code example or use a template; there are a large number of examples you could try, but for the purposes of this demonstration select an Empty Schematic, and choose workspace & project name you like; I’ve used ‘blink_led’.

Schematic

Now you’re faced with a blank circuit diagram, and may feel completely out of your comfort zone; why is a schematic needed, when all you want to do is blink an LED in software? The answer is that a PSoC, unlike a normal CPU, has really flexible peripherals, with no real pre-programmed function. So until you specify what you want by placing a component in a circuit diagram (the ‘TopDesign schematic’), the functionality doesn’t exist.

psoc_creator2
Blank schematic

To give you a quick guide to this screen, on the left you’ll see various design-wide resources that can be specified, such as I/O pins and the CPU clock. There are tabs for ‘Source’ and ‘Components’; you’ll frequently be switching between these views of your design. On the right side is the Component Catalogue containing various (soft) components that can be dragged onto the schematic.

The lower bar shows the result of compilation & device programming, as with many IDEs. The other area of note is the narrow menu bar to the left of the schematic; this selects drawing tools, the most important is the second one down, which is used to wire up the components.

Select a ‘Digital Output Pin’ from ‘Ports and Pins’ in the catalogue, and drag it onto the schematic. Zoom in so it is visible (I use control-mousewheel to do this). The default name is Pin_1 since all resources are auto-numbered, so double-click it and rename to LED_pin. You’ll see all sorts of other options, hinting at the extent to which everything is programmable; the only change you need to make is to un-check the ‘HW connection’ checkbox.

psoc_creator5
Configuring LED pin

It seems strange to have a hardware pin with no hardware, but what you are actually doing is removing the need for a hardware connection within the programmable logic, since we want the pin to be driven by software only.

psoc_creator3
Schematic with LED pin

You’ll also see a prompt to open the datasheet; try this, and you’ll be pleasantly surprised at the amount of information in the PDF. All components are documented in this way, which is one of the major strengths of the Cypress software tools.

Having defined an output to drive the LED, we need to link that output to a real hardware pin on the device. This is done by double-clicking ‘Pins’ in the Design Wide Resources, which displays a pretty little diagram of the device, with a hint that the LED_pin needs to be assigned.

psoc_creator4
Hardware pin assignment

Clicking the drop-down Port menu gives a long list of possible candidates; according to the kit packaging the LED is on port P2_1, so that is what we’ll select.

Before writing code, we need to define the CPU clock; double-click ‘Clocks’ in the design-wide resources, and you’ll see a spreadsheet of all the clocks we currently have.

psoc_creator6
Clock settings

The BUS_CLK needs to be defined, so double-click it and an even prettier flowchart pops up.

psoc_creator7
Clock flowchart

The IMO is the primary internal oscillator, and in the above settings I’ve used the PLL to boost it from 3 to 50 MHz. In theory the CPU can go up to 80 MHz, but then the timing analyser complains about the clock tolerance; I guess it implies that boosting an internal 3 MHz signal up to the CPU maximum isn’t a good idea, so I’ve backed it off to 50 MHz.

Back to the circuit diagram, right-click the LED pin, and open the datasheet; scroll through this remarkably long document until you get to the API definitions, and you’ll see that the function Pin_Write(uint8 value) is used to drive it. We’ll be using that function in the software.

Software

The source file ‘main.c’ was automatically created with the project; double-click it to open, and you’ll see a minimal framework.

One surprise is that there is a #include of the file ‘project.h’, with an error message saying it doesn’t exist. Don’t worry about it; this file contains the hardware definitions that are needed by the software, and will be generated at build-time.

The simplest way to add a delay is to use the built-in functions, so main.c becomes:

#include "project.h"

int main(void)
{
    CyGlobalIntEnable;
    for(;;)
    {
        LED_pin_Write(1);
        CyDelay(500);
        LED_pin_Write(0);
        CyDelay(500);
    }
}

The datasheet said the API function was Pin_Write, so why is it now called LED_pin_Write? This is an unusual aspect of PSoC programming, caused by the very tight integration between the hardware & software definitions of a device. At build-time, the system creates a custom version of the API to match your device settings in the schematic; the standard prefix ‘Pin’ is replaced with the name we provided, so Pin_Write becomes LED_pin_Write. Quite unconventional, but you soon get to appreciate the convenience of having a single shared namespace for the hardware & software.

In the Build menu, click ‘Build blink_led’ and watch as a myriad of files are generated, hopefully with no errors. If you see an error ‘No input on Instance “LED_pin”‘ then you didn’t disable the ‘HW connection’ on the LED pin (see above), so the software is expecting it to be wired to some internal logic.

Connect the Kitprog USB interface on the left-hand side of the board, and select Program in the Debug menu. If this is the first time you’ve used the board, you may be prompted to select it from a list.

After a brief flurry of programming, the blue LED on the board should start flashing at 1 Hz.

Hardware-only version

We’ll now use the PSoC’s programmable hardware to do the same task, as a jumping-off point for your own experimentation. There are various ways to do this, I’ve chosen one that demonstrates how components are wired up in the schematic.

First you need to double-click the LED pin on the schematic, and re-enable ‘HW connection’ so we can drive it from internal logic.

Now we need to create a suitable clock frequency; we’ll start with a 1 MHz signal, and divide it down to 1 Hz for the LED. Find the ‘Clock’ component in the System section of the Component Catalog, drag it onto the schematic, double click it, and use the following settings.

psoc_creator8
Clock settings

Now find the Frequency Divider in the Utility section of the Component Catalog, and drag it onto the schematic. Double-click, and set a divisor of 1000000.

psoc_creator9
Schematic components

All components follow the convention of having inputs on the left, outputs on the right; this part has Enable, Reset and Clock inputs. These have to be connected to something, you can’t leave inputs unconnected. Since we want the divider to be permanently enabled, and we don’t want it to be reset, we need to force Enable high, and Reset low, so find Logic High and Logic Low in the Digital Logic section of the Component Catalog, and drag one of each onto the schematic.

Now join the component pins using the wiring tool, which is the second one down in the left margin of the schematic; click on a pin, and the wire extends to the cursor; click on another pin, and they are joined. The end result should look something like:

psoc_creator10a
Completed schematic

Once joined, you can drag a component and it will remain connected; to delete a connection, click on a wire section, and press Delete.

All that remains is to get rid of our software, since everything is being done in hardware; change main.c to:

#include "project.h"
int main(void)
{
    for(;;)
    {
    }
}

Now rebuild the project, and program the device. You’ve created your first PSoC hardware project, and hopefully I’ve convinced you to carry on experimenting with these remarkable devices.

Copyright (c) Jeremy P Bentham 2019. Please credit this blog if you are using the information or software in it.

Creating real-time Web graphics with Python

Drawing graphics in browser-friendly SVG, with Javascript animation

My previous attempt at real-time graphics involved direct-drawing the shapes using Python and PyQt. This works well for a standalone system, but is less useful in the current world of ‘everything-on-a-browser’. To make the graphics Web-friendly, there is a choice; either create an animated bitmap (e.g. GIF, PNG or JPEG) or use vector graphics (SVG).

Vector graphics have various advantages; not just that they can be resized without degradation (pixellation) but also that they can preserve some of the structural characteristics of the image that is being animated. For example, we can draw a detailed circuit diagram complete with voltage and current measurements, and if it is too large to display in a browser screen, the user can zoom out to get the big picture, then zoom in to see the data in a specific area; the components will always appear well-drawn, no matter what scale is used.

qtraction_whole2
Animated circuit diagram
qtaction_part
Zoomed in to show voltage & current values

If the components are drawn sensibly they can easily be annotated & animated to reflect the current data values. Unfortunately, although most graphics packages can export SVG, very few preserve the underlying structure of the parts. For example, the above circuit diagram was originally exported from Visio as an 800K file, filled with tiny vector & text fragments, that were very difficult to relate back to the original components. The final version, created by a Python program from a netlist, was 33KB in size, and much easier to animate.

I certainly wouldn’t give up on the idea of animating pre-existing circuit diagrams, but for the purposes of this blog, am taking the easy way out, and creating the graphics from scratch in Python. Also, it is quite a big step to create & animate a full circuit diagram in one go, so I’ll start with something much simpler, a 7-segment digital clock.

seg_clock
Clock display in Web browser

This will involve Python, SVG, style sheets and even a smattering of Javascript, so is complicated enough for starters. If you are going to do a lot of graphical manipulation, it is well worth reading the SVG specification.

svgwrite

My code is compatible with Python 2.7 or 3.x, you just need to install the ‘svgwrite’ package using pip or pip3.

As its name suggests, svgwrite is a relatively slim wrapper that simplifies the task of writing SVG files, for example:

# Create simple SVG
import svgwrite
FILENAME      = "simple.svg"
WIDTH, HEIGHT = 250, 150

dwg = svgwrite.Drawing(FILENAME, (WIDTH, HEIGHT))
dwg.add(dwg.rect((0,0),
                 (WIDTH-1,HEIGHT-1),
                 stroke="red",
                 fill="none"))
dwg.add(dwg.line((0,0),
                 (WIDTH-1,HEIGHT-1),
                 stroke="green",
                 stroke_width=8))
dwg.add(dwg.circle((WIDTH/2,HEIGHT/2),
                   HEIGHT/2,
                   stroke_width=3,
                   stroke="salmon",
                   fill="moccasin"))
dwg.add(dwg.circle((WIDTH/2,HEIGHT/2),
                   HEIGHT/3,
                   stroke_width=3,
                   stroke="salmon",
                   fill="moccasin"))
dwg.save()

If the resulting SVG file is dragged into a browser, it looks like:

simple1

There are various points to note (apart from the author’s strange colour choices):

  • The origin (x=0, y=0) is in the top left corner
  • Coordinates are specified as (x,y) tuples
  • The order of drawing is significant: earlier objects may be obscured by later
  • Lines outside the drawing area are clipped
  • To eliminate fill, you have to use ‘fill=”none”‘, rather than the more Pythonic ‘fill=None’

It is instructive to view the resulting file in a text editor; after the namespace boilerplate, there is a close match with the Python code:

<?xml version="1.0" encoding="utf-8" ?>
<svg baseProfile="full" height="150" version="1.1" width="250" 
 xmlns="http://www.w3.org/2000/svg" 
 xmlns:ev="http://www.w3.org/2001/xml-events"
 xmlns:xlink="http://www.w3.org/1999/xlink">
 <defs />
 <rect fill="none" height="149" stroke="red" width="249" x="0" y="0" />
 <line stroke="green" stroke-width="8" x1="0" x2="249" y1="0" y2="149" />
 <circle cx="125" cy="75" fill="moccasin" r="75" stroke="salmon" stroke-width="3" />
 <circle cx="125" cy="75" fill="moccasin" r="50" stroke="salmon" stroke-width="3" />
</svg>

Style

The most obvious issue is the repetition of the colour definitions; using a style sheet would shorten the file, and separate the style from the drawing commands, making it easier to change the colours, for example:

import svgwrite

FILENAME      = "simple.svg"
WIDTH, HEIGHT = 250, 150

STYLES = """
  .circle_style  {stroke-width:3; stroke:salmon; fill:moccasin}
"""

dwg = svgwrite.Drawing(FILENAME, (WIDTH, HEIGHT))
dwg.defs.add(dwg.style(STYLES))
dwg.add(dwg.rect((0,0),
                 (WIDTH-1,HEIGHT-1),
                 stroke="red",
                 fill="none"))
dwg.add(dwg.line((0,0),
                 (WIDTH-1,HEIGHT-1),
                 stroke="green",
                 stroke_width=8))
dwg.add(dwg.circle((WIDTH/2,HEIGHT/2),
                   HEIGHT/2,
                   class_="circle_style"))
dwg.add(dwg.circle((WIDTH/2,HEIGHT/2),
                   HEIGHT/3,
                   class_="circle_style"))
dwg.save(pretty=True)

The browser display is exactly the same, and the SVG code now has an entry in the ‘defs’ section, with a strange addition:

<![CDATA[
  .circle_style {stroke-width:3; stroke:salmon; fill:moccasin}
]]></style>

The CDATA encapsulation is a signal to the browser that the normal search for HTML tags should be disabled; it isn’t really necessary in this case, but is essential if, for example, you are inserting a Javascript comparison such as ‘a < b’, since the browser would normally treat ‘<‘ as the start of an HTML tag.

Two important points:

  1. Note the underscore on the end of ‘class_’ in the Python code. If you forget that, Python will error out, as ‘class’ is a reserved word; svgwrite automatically strips off the trailing underscore when writing the SVG file.
  2. If you make any errors in the definition, the whole class will be ignored, and the object will be drawn in the default style.

The first issue is very common; you will see many complaints from svgwrite users that they can’t assign a class. The second issue can be annoying, since a trivial mistake can produce an ugly (or invisible!) mess, for example the following display results from putting the colour names in quotes (stroke:”salmon”; fill:”moccasin”).

style_error

The ‘developer tools’ mode in Chrome has highlighted the incorrect colour definitions.

Grouping

A useful SVG feature is that several items can be grouped together and treated as one, for example, instead of adding the 2 circles directly to the drawing, they can be added to a group, which is then added to the drawing:

g = svgwrite.container.Group(class_="circle_style")
g.add(dwg.circle((WIDTH/2,HEIGHT/2), HEIGHT/2))
g.add(dwg.circle((WIDTH/2,HEIGHT/2), HEIGHT/3))
dwg.add(g)

This is of most use when handling a complex shape, as all grouped items will be moved as one; if you want to move the 2 grouped circles to the right by 10 pixels, the first line becomes:

g = svgwrite.container.Group(class_="circle_style", transform="translate(10 0)")

You might think that the ‘add’ function would allow an offset to be applied to the items, but that isn’t true; it simply inserts the item into the file.

Using predefined items

If the the same element appears 2 or more times in your graphics, it is worthwhile defining it in the ‘defs’ section, then just adding it to the drawing with a ‘use’ command, which also allows the predefined graphic to be positioned and styled as required.

g = svgwrite.container.Group()
g.add(dwg.circle((WIDTH/2,HEIGHT/2), HEIGHT/2))
g.add(dwg.circle((WIDTH/2,HEIGHT/2), HEIGHT/3))
dwg.defs.add(g)
dwg.add(dwg.use(g, class_="circle_style"))
dwg.add(dwg.use(g, insert=(40,0), style="stroke:green; fill:palegreen"))
dwg.save()

simple4

You’ll note that I’ve removed the class from the group definition, and instead applied class & style definitions when the graphic is used. This isn’t compulsory; you can specify a class when the group is defined, but then it remains fixed, which limits the flexibility of your predefined objects – once a graphic is defined you can’t change its underlying structure.

It can also be really useful to name the predefined items using an ‘id’ parameter; they can then be inserted into the document by referencing that name, improving readability.

g = svgwrite.container.Group(id="circles")
g.add(dwg.circle((WIDTH/2,HEIGHT/2), HEIGHT/2))
g.add(dwg.circle((WIDTH/2,HEIGHT/2), HEIGHT/3))
dwg.defs.add(g)
dwg.add(dwg.use("#circles", class_="circle_style"))

The ‘#’ prefix is necessary to turn the ID into an Internationalised Resource Identifier (IRI).

Colour gradient

The green line looks a bit boring, so I wanted to add a colour gradient, making it look 3-dimensional. All the examples I’ve seen are of horizontal or vertical boxes, but I needed the shading to be aligned along the axis of the diagonal line, so it looks like a round bar. My intention was to create some very clever code to do this, but the result always looked terrible, so in the end I just hacked the parameters to make it look roughly right. Sorry!

lg = svgwrite.gradients.LinearGradient(
     start=(0,0), end=(40,0),
     id="blue_grad", gradientUnits="userSpaceOnUse",
     gradientTransform="rotate(120)")
lg.add_colors(['blue','white'])
dwg.defs.add(lg)
dwg.add(dwg.line(
    (0,0), (WIDTH-1,HEIGHT-1),
    stroke="url(#blue_grad)",
    stroke_width=50))
dwg.save()

simple5

Modifying objects

In the above code, we’ve been creating a graphic object and and adding it to the drawing in a single line. Alternatively, we can access and modify the object’s attributes using its dictionary, for example changing the stroke colour:

line = dwg.line((10,0), (110, 100), stroke="red")
...
# Change to green using either..
line["stroke"] = "green"
# ..or..
line.update({"stroke":"green"})
...
dwg.add(line)

Javascript

The intention is to create a working seven-segment clock display; we could possibly do this in Python by continually updating an SVG file, and persuading the browser to reload it, but a much simpler method is to use Javascript.

Each object (or group of objects) to be animated must have a unique ‘id’ tag.

dwg.add(dwg.line((20,0), (120, 100), stroke="red", id="myline"))

To change this line, we have to wait until the browser has loaded the graphics, so the code is triggered by window.onload event:

SCRIPT = """
window.onload = function()
{
    var myline = document.getElementById("myline");
    if (myline)
        myline.setAttribute("stroke", "yellow");
}
"""
dwg.defs.add(dwg.script(content=SCRIPT))

That’s how easy it is to modify a graphic at run-time using Javascript.

If you are a newcomer to that language, the developer’s console in the browser is incredibly useful; you can add diagnostic printouts using console.log(), and they will only show up when the developer tools are enabled. Here is the Chrome console when ‘console.log(myline);’ has been added to Javascript; hovering the cursor over the text display causes the associated graphic bounding box to be highlighted:

simple5_console

SVG clock

I started off by drawing the segments using simple flat-colour lines:

svg_clock

However the colour gradients look much more attractive..

seg_clock

..so I decided to use them instead. This triggered a cascade of problems, because using colour gradients on lines is much more restrictive than rectangles, in respect of the overall dimensions the browser uses to calculate gradient values (see the documentation on gradientUnits); when lines are drawn in various on-screen positions using the same gradient, only one is the correct colour.

The way round this problem was to draw only one shaded line, then use the ‘transform’ function to rotate and move a copy of that line. The parameters for the transformation are in the form of a 6-value array, which allows complex changes to be made in one step. Simplistically, the following transformations can be made:

(1, 0, 0, 1, x, y)          Move (translate) by x,y
(sx, 0, 0, sy, 0, 0)        Scale by sx, sy
(cos, sin, -sin, cos, 0, 0) Rotate by angle
(1, 0, tan, 1, 0, 0)        x-skew by angle
(1, tan, 0, 1, 0, 0)        y-skew by angle

A transformation array is needed for each of the 7 segments, so if segment A is the original drawn line:

segments

..the 7 segment transformations are..

# Transform matrix for each segment
SEG_MATRIX = ((1, 0,  0, 1,      0,        0), # A
              (0, 1, -1, 0, SEGLEN,        0), # B
              (0, 1, -1, 0, SEGLEN,   SEGLEN), # C
              (1, 0,  0, 1,      0, SEGLEN*2), # D
              (0, 1, -1, 0,      0,   SEGLEN), # E
              (0, 1, -1, 0,      0,        0), # F
              (1, 0,  0, 1,      0,   SEGLEN)) # G

The resulting lines are in a rectangular grid; the final flourish is to use a ‘skew’ transformation on the complete digit to give it a lean to the right:

# Matrix to slightly skew the displays
SKEW_MATRIX = "matrix(1,0,-0.1,1,0,0)"
...
g = Group(transform=SKEW_MATRIX)

The remaining question is how to do the animation; the simplest (and hopefully fastest) method I could think of is to stack all the digits 0 – 9 on top of each other, setting them transparent (opacity = 0). Each one is labelled with its position on the display, and its value, so ‘digit00’ is the left-most digit, with a value of 0, ‘digit01’ is the left-most digit with a value of 1, ‘digit10’ is the second digit with a value of 0, and so on. All the Javascript code has to do is set the opacity on the required digits, so to display the number 2345, digit02, digit13, digit24, and digit35 are opaque, the rest are transparent.

Source code

Here is the entire source code, compatible with Python 2.7 or 3.x

To view the resulting SVG file, click here.

# SVG clock with 7-segment display

import svgwrite
from svgwrite.container import Group

# 7-seg display
SEGWID      = 5             # Width & length of a segment
SEGLEN      = 20
NDIGITS     = 6             # Number of digits
DIG_PITCH   = 35            # Pitch of digits
DIG_SCALE   = 1             # Scaling factor for main digits
FRAC_SCALE  = 0.7           # Scaling factor for fractional part
LMARGIN     = 20            # Left margin
VMARGIN     = 5             # Vertical margins

GRAD_COLOURS= 'lightblue','blue'# Gradient colours
FNAME       = "svg_clock.svg"   # Filename

DWG_SIZE    = DIG_PITCH*NDIGITS, (VMARGIN+SEGLEN)*2

# Transform matrix for each segment
SEG_MATRIX = ((1,0, 0,1,0,      0),        # A
              (0,1,-1,0,SEGLEN, 0),        # B
              (0,1,-1,0,SEGLEN, SEGLEN),   # C
              (1,0, 0,1,0,      SEGLEN*2), # D
              (0,1,-1,0,0,      SEGLEN),   # E
              (0,1,-1,0,0,      0),        # F
              (1,0, 0,1,0,      SEGLEN))   # G

# Matrix to slightly skew the displays
SKEW_MATRIX = "matrix(1,0,-0.1,1,0,0)"

# Matrix to scale a segment down to a decimal point
DP_MATRIX   = "matrix(%g,0,0,1,0,0)" % (float(SEGWID)/SEGLEN)

# Segments to be activated for digits 0 - 9
dig_segs = ("abcdef", "bc", "abdeg", "abcdg", "bcfg",
            "acdfg", "acdefg", "abc", "abcdefg", "abcdfg")

SCRIPT = """
    var ndigits=6, last;

    // Initialise
    window.onload = function()
    {
        last = 0;
        setInterval(tick_handler, 100)
    }

    // Timer tick handler
    function tick_handler()
    {
        var now = new Date();
        var t = now.getHours()*10000 + now.getMinutes()*100 +
                now.getSeconds();
        if (t != last)
            set_value(t, last);
        last = t;
    }

    // Set a value on the display
    function set_value(val, oldval)
    {
        var i;
        for (i=0; i<ndigits; i++)
        {
            set_digit(ndigits-1-i, oldval, 0);
            set_digit(ndigits-1-i, val, 1);
            val = Math.floor(val / 10);
            oldval = Math.floor(oldval / 10);
        }
    }

    // Set opacity of a single display digit
    function set_digit(dig, val, op)
    {
        var digit = document.getElementById("digit"+dig%10+val%10);
        if (digit)
            digit.setAttribute("opacity", op);
    }
"""

# Create maximised SVG drawing
def create_svg(fname, size):
    return svgwrite.Drawing(fname,
            width="100%", height="100%",
            viewBox=("0 0 %u %u" % size),
            debug=False)

# Add Javascript to drawing
def add_script(dwg, script):
    dwg.defs.add(dwg.script(content=script))

# Create linear gradient, add to drawing defs
def create_lin_grad(dwg, id, start, end, colours):
    grad = svgwrite.gradients.LinearGradient(
           start=start, end=end,
           id=id, gradientUnits="userSpaceOnUse")
    grad.add_colors(colours)
    dwg.defs.add(grad)

# Create a single segment, add to drawing defs
def create_seg(dwg, id, length, width, stroke):
    seg = dwg.line((0,0), (length,0), id=id,
                   stroke="url(#%s)" % stroke,
                   stroke_width=width, stroke_linecap="round")
    dwg.defs.add(seg)

# Create digits 0 - 9 by transforming a segment, add to drawing defs
def create_digits(dwg, digid, segid):
    for dig, segs in enumerate(dig_segs):
        g = Group(id="%s%u" % (digid,dig), transform=SKEW_MATRIX)
        for seg in segs:
            num = ord(seg) - ord('a')
            mat = SEG_MATRIX[num]
            g.add(dwg.use('#'+segid, transform="matrix%s" % str(mat)))
        dwg.defs.add(g)

# Create decimal point by scaling a segment, add to drawing defs
def create_dp(dwg, dpid, segid):
    g = Group(id=dpid, transform=SKEW_MATRIX)
    g.add(dwg.use('#'+segid, transform=DP_MATRIX))
    dwg.defs.add(g)

# Add seven-segment display
# Each digit is a group of coincident transparent chars 0 - 9
def add_display(dwg, pos, pitch):
    dpos = list(pos)
    for n in range(0, NDIGITS):
        scale = DIG_SCALE if n<4 else FRAC_SCALE
        g = Group(transform = "translate(%g %g) scale(%g)" %
            (dpos[0], dpos[1], scale))
        for i in range(0, 10):
            g.add(dwg.use("#dig%u"%i, id="digit%u%u"%(n,i), opacity=0))
        dwg.add(g)
        if n == 2:
            dwg.add(dwg.use("#dp", insert=(dpos[0]-SEGLEN*0.7,
                                           dpos[1]+SEGLEN*2)))
        dpos[0] += pitch * scale

if __name__ == '__main__':
    dwg = create_svg(FNAME, DWG_SIZE)
    add_script(dwg, SCRIPT)
    create_lin_grad(dwg, "blue_grad", (0,0), (0,SEGWID*3/4),
                    GRAD_COLOURS)
    create_seg(dwg, "seg", SEGLEN, SEGWID, "blue_grad")
    create_digits(dwg, "dig", "seg")
    create_dp(dwg, "dp", "seg")
    add_display(dwg, (LMARGIN, VMARGIN), DIG_PITCH)
    dwg.add(dwg.text("iosoft.blog",
            (LMARGIN-2+DIG_PITCH*4, DWG_SIZE[1]-5),
            style="font-size:8px; font-family:Arial", fill="gray"))
    dwg.save(pretty=True)

# EOF

Copyright (c) Jeremy P Bentham 2019. Please credit this blog if you use the information or software in it.

Raspberry Pi and OpenOCD

In previous blog posts I used an FTDI module and pure Python code to access the internals of an ARM CPU using the SWD interface. I want to expand this technique to provide a more comprehensive real-time display of the CPU status, but the FTDI interface is quite limiting; what I need is an fast intelligent SWD/JTAG adaptor, with a network interface so I can do both local and remote diagnosis.

Enter the raspberry Pi: a lot of computing power at very low cost, either using the built-in HDMI display output, or running ‘headless’ over a wireless network, providing diagnostic data to a remote display.

Connecting the Pi to the target system could hardly be simpler; 3 wires (clock, data & ground) are sufficient to access data from most CPUs with an SWD interface.

rpi3_swd
Raspberry Pi 3 SWD interface to STMicro ARM CPU
jtag_sam7
Raspberry Pi ZeroW JTAG interface to Atmel ARM CPU

Software-wise, OpenOCD has all the SWD/JTAG features you’ll ever need, accessed through a network interface; installation may be a bit intimidating if you’re not an experienced Linux user, but is really quite easy, as this blog will (hopefully) demonstrate.

What you end up with is a really powerful local/remote debugger for very little money; around $10 US, in the case of the Pi Zero W.

Installing OpenOCD

You need any Raspberry Pi (RPi), versions 0 to 3. The slower boards will have longer boot times & build times but are otherwise fully functional. The OS version I used was ‘Raspbian Stretch with desktop’; the ‘recommended software’ add-ons are not necessary. The total image size on SD card is around 3 GB.

A convenient way to avoid re-typing the instructions below is to enable the Secure Shell (SSH) protocol using the ‘Raspberry Pi Configuration utility’, then run a remote ssh client (e.g. ‘putty’ on Windows) to access the RPi over the network; you can then cut and paste a command line into the ssh window without re-typing. If in doubt as to the IP address of your RPi, hover the cursor over the network icon in the top-right corner, and the address will be shown, e.g. 192.168.1.220 (or run ‘ifconfig’ if in text mode).

It is best to install OpenOCD from source, as the pre-built images often lack important functionality. Installation instructions can be found on many Web sites, for example Adafruit “Programming Microcontrollers using OpenOCD on a Raspberry Pi”. In summary, the steps are:

cd ~
sudo apt-get update
sudo apt-get install git autoconf libtool make pkg-config libusb-1.0-0 libusb-1.0-0-dev
git clone http://openocd.zylin.com/openocd
cd openocd
./bootstrap
./configure --enable-sysfsgpio --enable-bcm2835gpio
make
sudo make install

The ‘make’ step takes approximately an hour on the slower boards, or 15 minutes on the faster.

Configuration files

OpenOCD has a wide variety of options, so generally needs more than one configuration file, to define:

  • Debug adaptor (in our case, the RPi)
  • Communication method (SWD or JTAG)
  • Target CPU.

There are a large number of files in /usr/local/share/openocd/scripts, most notably the ‘interface’ and ‘target’ sub-directories, however there are so many permutations that it is unlikely you’ll find everything you need, so we need to think about creating our own files.

The most important first step is to work out how the RPi will be connected to the target system…

RPi I/O connections

At the time of writing, there are 3 versions of the RPi I/O connector, and 3 different pin-numbering schemes, so it is easy to get confused. The older boards may be considered obsolete, but are still more than adequate for running OpenOCD, so mustn’t be excluded.

The numbering schemes are:

  1. Connector pin numbers: sequential 1 – 26 or 1 – 40
  2. GPIO bit numbers (also known as Broadcom or BCM numbers) 0 – 27
  3. WiringPi numbers, as used in the Python library

I’ll only be using the first 2 of these. The older boards have 26 pins, the newer 40.

rpi1_gpio
RPi 26-way connector with GPIO numbers

Pins 3 & 5 were initially GPIO 0 and 1, but later became GPIO 2 and 3; they are best avoided.

rpi2_gpio
RPi 40-way connector with GPIO numbers

On the 40-way connector GPIO21 has become 27, so should also be avoided. The choice of ground pin is arbitrary; any of them can be used, but I avoid pin 6, as any mis-connection to the supply pins can result in significant damage.

SWD

The SWD connections given in the OpenOCD configuration file ‘raspberrypi2-native.cfg’ are:

rpi_swd
raspberrypi2-native SWD connections

The relevant lines in the configuration file are:

# SWD                 swclk swdio
# Header pin numbers: 22    18
bcm2835gpio_swd_nums  25    24

bcm2835gpio_srst_num  18
reset_config srst_only srst_push_pull

In many applications the reset signal is unnecessary – and undesirable, if the objective is to perform non-intrusive monitoring of a running system.

JTAG

JTAG is an older (and more widely available) standard for debugging, that requires 4 wires in the place of 2 for SWD. There is a standard mapping between them (SWCLK is TCK, SWDIO is TMS), but the JTAG connections in the standard OpenOCD configuration file ‘raspberrypi-native.cfg’ use completely different pins:

rpi_jtag
raspberrypi-native JTAG connections

The relevant lines in the configuration file are:

# JTAG                tck tms tdi tdo
# Header pin numbers: 23  22  19  21
bcm2835gpio_jtag_nums 11  25  10  9

# bcm2835gpio_srst_num 24
# reset_config srst_only srst_push_pull

As standard, the reset definition is commented out.

I’m not a fan of this pinout scheme; I’d like a single setup that covers both SWD and JTAG.

Other pin functions

You might wish to use the RPi for other diagnostic functions, such as monitoring a serial link, so these pins have to be kept free. The following diagram shows the alternative pin functions.

rpi_pinout2

You can use any of the blue or yellow pins for the SWD/JTAG interface, it is just a question as to which other functionality you may be needing.

Combining SWD and JTAG

The compromise I’ve adopted is to preserve the existing SWD arrangement, but move the JTAG pins so one set of connections can serve both SWD & JTAG on either the 26-way or the 40-way connectors – and I’ve also avoided using any of the predefined pins, so there are no conflicts with other functionality.

rpi_swd_jtag
rpi_swd_jtag SWD and JTAG connections

The relevant section of the configuration file is:

# SWD                swclk swdio
# Header pin numbers 22    18
bcm2835gpio_swd_nums 25    24

# JTAG                tck tms tdi tdo
# Header pin numbers  22  18  16  15 
bcm2835gpio_jtag_nums 25  24  23  22

Target system connections

The connection points on the target system will vary from board to board; for a previous demonstration I used a ‘blue pill’ STM32F103 board that has ground, SWD clock & data conveniently on some separate header pins, but the most common standard for JTAG & SWD connections is a 20-way 2-row header, as follows:

JTAG     SWD     20-way pin
Ground   Ground  4, 6, 8, 10, 14, 16, 18, 20
TRST             3
TDI              5
TMS      SWDIO   7
TCK      SWCLK   9
TDO              13
RESET            15

There is generally a keyway on the odd-numbered side of the connector.

swd_jtag_conn

Two reset signals are defined: TRST is ‘tap reset’, that is intended to just reset the diagnostic port; the other signal marked RESET (which OpenOCD refers to as SRST or ‘system reset’) should reset all devices, as if a reset button has been pressed. In the experimentation I’ve done, the reset lines haven’t been needed, but this is very processor-specific; sometimes the RESET line has to be used to gain control of the target system.

It is convenient to use ribbon cable for wiring up the interface, especially if the wires follow the resistor colour code:

RPi pin  Colour  20-way pin  JTAG/SWD
9        Brown   20          Ground
12       Red     15          Reset
16       Orange  5           TDI
15       Yellow  13          TDO
18       Green   7           TMS/SWDIO
22       Blue    9           TCK/SWCLK

Or in graphical form…

swd_jtag_wiring

Interface configuration file

The above examples show how the SWD/JTAG connections are handled, but some more data is needed to fully configure the RPi interface, most notably the I/O base address and clock scaling; this tells OpenOCD where to find the I/O interface, and how to compute its speed.

There are 2 possible values for the I/O base address: the RPi zero and v1 use 0x20000000, and v2+ use 0x3F000000. If you are unsure which value to use, the boards have an excellent feature called Device Tree that documents the current hardware configuration; enter the following command in a console window:

xxd -c 4 -g 4 /proc/device-tree/soc/ranges

The base I/O address is the second value returned, for example:

RPi zero:
00000000: 7e000000  ~...
00000004: 20000000   ...
00000008: 02000000  ....
RPi v3:
00000000: 7e000000 ~...
00000004: 3f000000 ?..
00000008: 02000000 ....
..and so on..

The clock scaling is less critical, since we’re generally aiming for around 1 MHz, which gives quite a bit of leeway in terms of being fast or slow. This is fortunate, because it is difficult to find a definitive explanation of the values that should be used for all hardware & clock settings. My understanding, from reading the source code, is that every I/O read or write instruction is followed by a loop containing NOP (CPU idle) cycles to space out the operations; this number is known as the ‘jtag_delay’, and is calculated by:

(speed_coeff / khz) - speed_offset;

..where speed_coeff & speed_offset are the two scaling parameters, and khz is the desired SWD/JTAG clock speed in kHz (all the values are integers). Obviously the delay is very CPU-dependant; the standard values in the files are:

Rpi zero and v1:
  bcm2835gpio_speed_coeffs 113714 28
RPi v2+:
  bcm2835gpio_speed_coeffs 146203 36

These do seem to give roughly the right answers, and there isn’t any great necessity for the delays to be accurate – when viewed on an oscilloscope, you can see some of the cycles being stretched by an incoming interrupt, so they never will be as accurate as a pure hardware solution.

Adaptor configuration files

Combining all the information above, here are the two adaptor configuration files: rpi1.cfg for RPi zero & v1, and rpi2.cfg for v2+

# rpi1.cfg: OpenOCD interface on RPi zero and v1

# Use RPi GPIO pins
interface bcm2835gpio

# Base address of I/O port
bcm2835gpio_peripheral_base 0x20000000

# Clock scaling
bcm2835gpio_speed_coeffs 113714 28

# SWD                swclk swdio
# Header pin numbers 22    18
bcm2835gpio_swd_nums 25    24

# JTAG                tck tms tdi tdo
# Header pin numbers  22  18  16  15 
bcm2835gpio_jtag_nums 25  24  23  22
# rpi2.cfg: OpenOCD interface on RPi v2+

# Use RPi GPIO pins
interface bcm2835gpio

# Base address of I/O port
bcm2835gpio_peripheral_base 0x3F000000

# Clock scaling
bcm2835gpio_speed_coeffs 146203 36

# SWD                swclk swdio
# Header pin numbers 22    18
bcm2835gpio_swd_nums 25    24

# JTAG                tck tms tdi tdo
# Header pin numbers  22  18  16  15 
bcm2835gpio_jtag_nums 25  24  23  22

Running OpenOCD

Finally we get to run OpenOCD, but in addition to the adaptor configuration, we need to give some details about the interface & target CPU.

The command line consists of configuration files prefixed by -f, and commands prefixed by -c. In reality, a configuration file is just a series of commands; for example you can select JTAG operation using the command-line option:

openocd -c "transport select jtag"

This is exactly the same as:

openocd -f select_jtag.cfg

where the file ‘select_jtag.cfg’ has the line:

transport select jtag

So we’ll use a mixture of commands and files on our command line. The following example is for an RPi v3 driving an SWD interface into a STM32F103 processor;  I’ve used backslash continuation characters at the end of each line to make the commands more readable:

sudo openocd -f rpi2.cfg \
             -c "transport select swd" \
             -c "adapter_khz 1000" \
             -f target/stm32f1x.cfg

Some hardware operations require superuser privileges, hence the use of ‘sudo’. The usual security warnings apply when doing this; you can try without, there will just be a ‘permission denied’ error if it fails.

For a list of supported CPUs, see the files in /usr/local/share/openocd/scripts/target

When OpenOCD runs, with a bit of luck, you’ll see something like:

BCM2835 GPIO nums: swclk = 25, swdio = 24
BCM2835 GPIO config: tck = 25, tms = 24, tdi = 23, tdo = 22
swd
adapter speed: 1000 kHz
adapter speed: 1000 kHz
adapter_nsrst_delay: 100
none separate
cortex_m reset_config sysresetreq
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : BCM2835 GPIO JTAG/SWD bitbang driver
Info : JTAG and SWD modes enabled
Info : clock speed 1001 kHz
Info : SWD DPIDR 0x1ba01477
Info : stm32f1x.cpu: hardware has 6 breakpoints, 4 watchpoints
Info : Listening on port 3333 for gdb connections

If there is a configuration or wiring error, OpenOCD usually (but not always!) returns to the command line, for example if the SWDIO line is disconnected:

BCM2835 GPIO nums: swclk = 25, swdio = 24
BCM2835 GPIO config: tck = 25, tms = 24, tdi = 23, tdo = 22
swd
adapter speed: 1000 kHz
adapter speed: 1000 kHz
adapter_nsrst_delay: 100
none separate
cortex_m reset_config sysresetreq
Info : Listening on port 6666 for tcl connections
Info : Listening on port 4444 for telnet connections
Info : BCM2835 GPIO JTAG/SWD bitbang driver
Info : JTAG and SWD modes enabled
Info : clock speed 1001 kHz
Info : SWD DPIDR 0x02192468

..and then OpenOCD terminates back to the command line..

The clue is in the SWD Data Port ID Register (DPIDR) value. According to the datasheet for the STM32F103 CPU, this should be 1BA01477. With a data line fault, every time OpenOCD runs, a different value is returned, e.g. 0x00e65468, 0x02192468, 0x00433468 and so on; the software is just picking up noise on the data line.

A disconnected clock line is harder to diagnose, as OpenOCD just terminates after the ‘clock speed’ report, with no error indication. Try using the -d option to invoke a debug display, and you’ll see lines like

JUNK DP read reg 0 = ffffffff

which suggests that all is not well in the hardware interface.

Another thing to try in the event of a failure is adding or removing a reset line, and changing its configuration entries; if there is a reset problem you’ll probably see the DPIDR value reported correctly, but other functions may not work.

What now?

Having just written 2100 words and drawn 8 diagrams, I’m going to take a short break. However, first I ought to give some indication as to how you control this OpenOCD setup.

The sign-on text mentions a telnet interface on port 4444, so we can use that; the commands highlighted in bold:

sudo apt-get install telnet  # ..if not already installed

telnet localhost 4444
Trying ::1...
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Open On-Chip Debugger
>

As standard, this interface only works when telnet is running locally on the Raspberry Pi. To open it up to a wider network, add the command ‘bindto 0.0.0.0’ to the configuration. However, this option comes with a major security warning; think very carefully before making the system accessible to everyone on the network.

Refer to the OpenOCD documentation for information on the large number of commands that can be used over telnet, for example displaying memory using ‘mdw’ or halting the processor using ‘halt’. When finished, close the telnet link with ‘exit’.

If that doesn’t sound very exciting, you’ll have to wait for my next blog where I combine the  power of the Raspberry Pi with the graphics in my reporta project, to produce a non-invasive high-speed display of the CPU internals – a graphical real-time viewer for your embedded systems. Watch this space…

Copyright (c) Jeremy P Bentham 2019. Please credit this blog if you use the information or software in it.

Viewing ARM CPU activity in real time

viewing_cpu2

In previous blog posts, I have described how an FTDI USB device can be programmed in Python to access the SWD bus of an ARM microprocessor. This allows the internals of the CPU to be accessed, without disrupting the currently running program.

In this blog I take the process one step further, and add a graphical front-end, that shows the CPU activity in real time; if you want to see it in action, take a look at this video.

If you’re not a fan of FTDI hardware, or need a faster debug system, take a look at my post OpenOCD on the Raspberry Pi.

Hardware

The target system in the demonstration is a ‘blue pill’ STM32F103 board, with a 7-segment display and pushbutton. This CPU board is particularly convenient because it has a 4-pin connector with SWD clock & data; it can be seen on the right-hand side of the photograph above.

The SWD connection is described in detail in this post, in brief, the circuit is:

ftdi_cable
SWD cable circuit diagram

Take care when making any connections to power lines, especially the 5 volt line on the FTDI module; if mis-connected, high currents can flow, resulting in significant damage.

The demonstration system has a Kingbright SC56-11LGWA common-cathode 7-segment display; the anodes are driven directly from the CPU I/O pins with 220 ohm series resistors. The pin mapping (reading from top left of the display) is:

Segment   CPU Pin (via 220R)
g         PB11
f         PB10
a         PB1
b         PB0
e         PB12
d         PB13
c         PB14
DP        PB15

Button    PB3 (button shorts this pin to ground)

You may wonder why I have used such a complex mapping; why not use 8 consecutive pins? The answer is that the above arrangement makes the wiring easier, at the expense of a little software complexity. This trade-off is quite common in commercial projects, where the demands of cost-saving often lead to significant complexity in the hardware configuration.

Software

reporta

The full source code is available on Github, and is compatible with Python 2.7 or 3.x. It has been tested on Windows, and is theoretically Linux-compatible, except for a problem reading data back from USB, as described in the earlier blog – this issue needs to be resolved. The Python library dependencies are:

  • PyQt v4 or v5 (GPL version)
  • ftd2xx
  • pypiwin32 (if running Python 3.x)

These can be installed with ‘pip’ or ‘pip3’ as usual.

PyQt

The following text describes how I created the graphical front-end, for the benefit of anyone wishing to understand or modify the code; this isn’t necessarily the best way, it is just what I did based on past experience.

There are many options for creating a GUI in Python; I’ve used PyQt in the past, so that is the option I’ve chosen here. In case you’re unfamiliar with it, the current version is 5, but many installations are still version 4, so I’ve written the code to be compatible with both, even though this does involve some manipulation of the libraries:

try:
    from PyQt5.QtGui import QBrush, QPen, QColor, QFont, QTextCursor, QFontMetrics, QPainter
    from PyQt5.QtWidgets import QApplication, QGraphicsScene, QGraphicsView, QGraphicsSimpleTextItem
    from PyQt5 import QtCore, QtWidgets
except:
    from PyQt4.QtGui import QBrush, QPen, QColor, QFont, QTextCursor, QFontMetrics, QPainter
    from PyQt4.QtGui import QApplication, QGraphicsScene, QGraphicsView, QGraphicsSimpleTextItem
    from PyQt4 import QtCore, QtGui as QtWidgets

It is possible to create the entire front-end graphically using Qt designer, then just import the GUI file and it will be displayed exactly as designed, so in theory you only need to write the event-handlers for the various actions. Personally, I find this approach a bit tricky when implementing more complex GUIs, so tend to use the PyQt function calls to build the graphics from scratch.

Main Window

The main window is as simple as possible, containing only one central widget:

class MyWindow(QtWidgets.QMainWindow, MyWidget):
    graph_updater = QtCore.pyqtSignal(str)

    def __init__(self, parent=None):
        QtWidgets.QMainWindow.__init__(self, parent)
        self.widget = MyWidget(self)
        self.setCentralWidget(self.widget)
        self.setWindowTitle(VERSION)
        self.resize(*WINDOW_SIZE)
        self.graph_updater.connect(self.widget.update_graph)

A ‘graph_updater’ signal is defined so that any thread can send a request to update the graphics; this will be queued until the main GUI thread is back in control, so there is no ambiguity over which thread is performing the updates.

The single widget contains all the graphical elements, and the hierarchy is important if  they are to be displayed correctly; for example, if the window is resized, I want all the elements to be resized in proportion, and that is only possible if the correct parent-child relationship is maintained.

The upper area of the widget is graphics, the lower is text; they are combined using a vertical layout widget.

self.text = QtWidgets.QTextEdit()
self.scene = QGraphicsScene()
self.view = MyView(self.scene)
...
layout = QtWidgets.QVBoxLayout()
layout.addWidget(self.view, 30)
layout.addWidget(self.text, 10)
self.setLayout(layout)

Text display

The text area is used to display all kinds of diagnostic information, so it is convenient to redirect the console print function to display here. This is done by creating ‘write’ and ‘flush’ functions in the widget, which emit a signal linked to an updater function:

text_updater = QtCore.pyqtSignal(str)
...
self.text_updater.connect(self.update_text)
...
# Handler to update text display
def update_text(self, text):
    disp = self.text.textCursor()           # Move cursor to end
    disp.movePosition(QTextCursor.End)
    text = str(text).replace("\r", "")      # Eliminate CR
    while text:
        head,sep,text = text.partition("\n")# New line on LF
        disp.insertText(head)
        if sep:
            disp.insertBlock()
    self.text.ensureCursorVisible()    # Scroll if necessary

# Handle sys.stdout.write: update text display
def write(self, text):
    self.text_updater.emit(str(text))
def flush(self):
    pass

Now all that is necessary is to redirect console output to the main widget..

sys.stdout = self

..and magically anything in a ‘print’ function will appear in the text display. The PyQt Signal interface ensures there are no threading problems, you can still use the print function anywhere.

Graphical display

When displaying graphics, Qt (and hence PyQt) makes a distinction between the graphical objects (the ‘scene’) and their realisation on the screen (the ‘view’).

When first experimenting with background patterns, I discovered one (Dense1Pattern) that creates rows of holes similar to the prototyping board. Having fixed on this, it was logical to draw all objects with respect to this grid, i.e. in nominal units of 0.1 inches as on the prototyping board, though the graphics will expand or shrink when the window size is changed.

Drawing

To draw an object, it is just added to the scene, and it will automatically be displayed, for example to draw a circle:

# Add circle to grid, given centre
    def draw_circle(self, gpos, size, pen, brush=PIN_ON_BRUSH):
        size *= GRID_PITCH
        x,y = self.grid_pos(gpos)
        p = self.scene.addEllipse(0, 0, size, size, pen, brush)
        p.setPos(x-size/2.0, y-size/2.0)
        return p

You can see the conversions from screen-units to grid-units. One less-obvious aspect of this code is that the ellipse is originally drawn at the 0,0 origin, then moved into place, rather than being drawn at the final location; this simplifies any subsequent operations such as movement or rotation.

Animation

The drawing function returns the object that has been created, which must be saved somewhere, so it can be animated.  The most obvious form of animation is to replace that object with another, e.g. one drawn in dark red to light red, but there is an easier way to modify any drawn object: change the opacity – the extent to which the object is transparent or opaque. If a bright red object is drawn with an opacity value of 0.1, it becomes a faint dark red; changing the opacity to 1.0 restores the full strong colour.

So the objects that are to be animated are stored as a list in a dictionary, indexed on the signal name (e.g. ‘PB10’); when that signal changes state, it is only necessary to walk the list, setting the opacity as required.

# Set pin (or segment) on/off state
# Format is 'name=value', e.g.  'PA10=1'
def set_pin(self, s):
    name, eq, num = s.partition('=')
    if eq and name in self.sigpins:
        val = int(num, 16)
        for p in self.sigpins[name]:
            if int(p.opacity()) != val:
                p.setOpacity(PIN_ON_OPACITY if val else PIN_OFF_OPACITY)

7-segment display

As it happens, PyQt has a widget to draw 7-segment displays, but in this case it is easier to animate if drawn from scratch. The standard segment notation is:

sevenseg

To draw this in one continuous operation, I start with segment F, then ABCDEG. Once drawn, the list is rearranged into ABCDEFGH order, where H is the decimal point.

SWD interface

The SWD interface uses an FTDI USB device as described in detail here. The software used in this project is very similar, but has been optimised to scan the I/O ports quite fast, around 100 times per second. The way this has been achieved is to group all the commands to the FTDI device together, so a single block of data requests is sent out over the USB bus, then a single block of responses is read back.

All the outgoing requests are buffered, rather than being sent individually – this is quite a simple change, you just have to remember to flush the buffer before reading back the results. However you then have the problem that the returned data block consists of the data from several requests – how do you work out which data value corresponds to which request? The method I’ve adopted is to create a polling list, with objects representing the memory addresses to be polled

poll_vars = []  # List of variables to be polled

# Storage class for variable to be polled
class Pollvar(object):
    def __init__(self, name, addr):
        self.name, self.addr = name, addr
        self.value = None

# Add variable to the polling list
def poll_add_var(name, addr):
    poll_vars.append(Pollvar(name, addr))

The data requests are generated by walking down the list, then when the responses arrive, the ‘value’ fields are filled in sequentially, or set to ‘none’ if there was an error.

# Send out poll requests
def poll_send_requests(h):
    for pv in poll_vars:
        swd.swd_wr(h, swd.SWD_AP, APORT_TAR, pv.addr, True, False)
        swd.swd_idle_bytes(h, 2)
        swd.swd_rd(h, swd.SWD_AP, APORT_DRW, True, False)
        swd.swd_rd(h, swd.SWD_AP, APORT_DRW, True, False)

# Get poll responses
def poll_get_responses(h):
    for pv in poll_vars:
        swd.swd_wr(h, swd.SWD_AP, APORT_TAR, pv.addr, False, True)
        swd.swd_rd(h, swd.SWD_AP, APORT_DRW, False, True)
        req = swd.swd_rd(h, swd.SWD_AP, APORT_DRW, False, True)
        pv.value = req.data.value if (req.data is not None and
                    req.ack.value==swd.SWD_ACK_OK) else None

It is slightly confusing that the request & response use the same swd_wr and swd_rd functions; the key to understanding this code is to look at the boolean values. ‘True, False’ means that a transmission is sent, but no data is read back; ‘False, True’ is the opposite, in that nothing is sent, the response is just read back.

If you want to see the SWD transactions, try setting swd.VERBOSE to True:

SWD interface: FT232H device in Single RS232-HS
Reporta

Rd 0 IDCODE  1BA01477 Ack 1
Wr 0 ABORT   0000001E Ack 1
Wr 4 CTRL    50000000 Ack 1
Rd 4 STATUS  F0000040 Ack 1
DP ident: 1BA01477
Wr 8 SELECT  000000F0 Ack 1
Rd C DRW/BD3 00003BDB Ack 1
Rd C DRW/BD3 14770011 Ack 1
AP ident: 14770011
Wr 8 SELECT  00000000 Ack 1
Wr 0 CSW/BD0 22000002 Ack 1
Wr 4 TAR/BD1 40010C08
Rd C DRW/BD3
Rd C DRW/BD3
Wr 4 TAR/BD1 40010C08 Ack 1
Rd C DRW/BD3 14770011 Ack 1
Rd C DRW/BD3 000043D9 Ack 1
..and so on..

Looking at the last block of 6 transactions, the first 3 are a write-cycle to the SWD TAR (transfer address) register, then a dummy read and the actaul read cycle. These are the requests being sent to the target system; nothing is being read back so the read-data & ack-status values are unknown. The second block of 3 are the readback of these requests, so the actual values are displayed.

An ‘ack’ value of anything apart from 1 is an error; a value of 0 or 7 suggests the target isn’t responding, 2 means it is trying to send a delayed response, and 4 indicates a hard error – this is sticky, so will persist until cleared by writing to the ‘abort’ register.

Code structure

The ‘reporta’ project source files on Github are:

reporta.py:   main program
rp_pyqt.py:   PyQt interface
rp_arm.py:    ARM processor definitions
rp_swd.py:    SWD interface
rp_ftd2xx.py: FTDI device driver

The lower 3 files are very similar to their counterparts in these posts, basically I’ve copied the files with minor modifications. Each file has a ‘main’ function to demonstrate its functionality; for example, you can see the PyQt interface running without the FTDI hardware, just by executing rp_pyqt.py

The main ‘reporta’ program has no command-line options, currently the I/O port of interest is hard-coded; to add other ports (or CPU memory locations) to the polling list, just use the poll_add_var function.

As with the previous posts, I must state that so far I’ve only tested this technique on the STM32F103 processors; others may require custom powerup sequences, or special incantations to gain access to the CPU memory – but hopefully my Python code will provide a useful starting-point.

Copyright (c) Jeremy P Bentham 2018. Please credit this blog if you are using the information or software in it.

Programming FTDI devices in Python: Part 5

Doing something useful with SWD

In part 4 we got as far as reading in the CPU identification, which is of no real use; in this part we’ll actually read some of the CPU internals, but first we need to understand how SWD accesses are controlled.

Exif_JPEG_PICTURE
SWD cable showing resistor

DAP: DP and AP

You may think that we just need to feed a 32-bit address into the SWD port, and get a value back, but the reality is much more complicated. The SWD clock & data lines are connected to the CPU Debug Port (SW-DP), which has its own address space. Read/write accesses to the CPU memory aren’t controlled by the DP; you have to go through the Access Port (AP), which also has a separate memory space, and is bank-switched.

DP

We’ve already accessed the Debug Port ID code register, the other read-write registers are:

# Debug Port (SWD-DP) registers
# See ARM DDI 0314H "Coresight Components Technical Reference Manual"
DPORT_IDCODE        = 0x0   # ID Code / abort
DPORT_ABORT         = 0x0
DPORT_CTRL          = 0x4   # Control / status
DPORT_STATUS        = 0x4
DPORT_SELECT        = 0x8   # Select
DPORT_RDBUFF        = 0xc   # Read buffer

These registers control various aspects of the debug interface, for example the ‘abort’ register is used to reset the internal logic after an error, and the ‘control’ register is used to power up the peripherals needed for debugging.

The ‘select’ register has bitfields to control AP and DP bank-switching; it is convenient to define this using CTYPES:

from ctypes import Structure, Union, c_uint
# AHB-AP Select Register
class DP_SELECT_REG(Structure):
    _fields_ = [("DPBANKSEL",   c_uint, 4),
                ("APBANKSEL",   c_uint, 4),
                ("Reserved",    c_uint, 16),
                ("APSEL",       c_uint, 8)]
class DP_SELECT(Union):
    _fields_ = [("reg",   DP_SELECT_REG),
                ("value", c_uint)]
dp_select = DP_SELECT()

The union allows us to specify the whole 32–bit register when doing SPI read & write cycles, but still access individual bitfields within it.

AP

The AP registers are:

# Access Port (AHB-AP) registers, high nybble is bank number
# See ARM DDI 0337E: Cortex M3 Technical Reference Manual page 11-38
APORT_CSW           = 0x0   # Control status word
APORT_TAR           = 0x4   # Transfer address
APORT_DRW           = 0xc   # Data read/write
APORT_BANK0         = 0x10  # Banked registers
APORT_BANK1         = 0x14
APORT_BANK2         = 0x18
APORT_BANK3         = 0x1c
APORT_DEBUG_ROM_ADDR= 0xf8   # Address of debug ROM
APORT_IDENT         = 0xfc   # AP identification

The Control Status Word register controls various aspects of the CPU memory accesses, for example the data size, and auto-increment for reading blocks of memory:

# AHB-AP Control Status Word Register
class AP_CSW_REG(Structure):
    _fields_ = [("Size", c_uint, 3),
                ("Res1", c_uint, 1),
                ("AddrInc", c_uint, 2),
                ("DbgStatus", c_uint, 1),
                ("TransInProg", c_uint, 1),
                ("MODE", c_uint, 4),
                ("Res2", c_uint, 13),
                ("HProt1", c_uint, 1),
                ("Res3", c_uint, 3),
                ("MasterType", c_uint, 1),
                ("Res4", c_uint, 2)]
class AP_CSW(Union):
    _fields_ = [("reg", AP_CSW_REG),
                ("value", c_uint)]
ap_csw = AP_CSW()

Because the AP can be accessing main CPU memory, it has two time-dependencies:

  • the AP may return a ‘wait’ indication in the status field so the transaction has time to go though
  • the data isn’t returned immediately; first you have to do a dummy read cycle, then a second read cycle that actually returns the data

So to set up a CPU memory read cycle, we need to configure the AP (including its bank-switching) then set a transfer address

# Configure AP memory accesses
def ap_config(d, inc, size):
    dp_select.reg.APBANKSEL = 0     # Zero bank
    swd.swd_wr(d, swd.SWD_DP, DPORT_SELECT, dp_select.value)
    ap_csw.reg.MasterType = 1
    ap_csw.reg.HProt1 = 1           # Enable incrementing, set access size
    ap_csw.reg.AddrInc = 1 if inc else 0
    ap_csw.reg.Size = 0 if size==8 else 1 if size==16 else 2
    return swd.swd_wr(d, swd.SWD_AP, APORT_CSW, ap_csw.value)

# Set AP memory address
def ap_addr(d, addr):
    swd.swd_wr(d, swd.SWD_AP, APORT_TAR, addr)
    swd.swd_idle_bytes(d, 2)            # Idle to avoid 'wait' response

# Do an immediate read of a 32-bit CPU memory location
def cpu_mem_read32(d, addr):
    ap_addr(d, addr)                              # Address to read
    swd.swd_rd(d, swd.SWD_AP, APORT_DRW)          # Dummy read cycle
    r = swd.swd_rd(d, swd.SWD_AP, APORT_DRW)      # Read data
    return r.data.value if r.ack.value==swd.SWD_ACK_OK else None

There are 2 idle (null) bytes after the target memory address is set. These give the AP time to process the address value before it is used; if omitted, the AP gives an ack value of 2 (‘wait’) on the next transaction.

User interface

The console-based user interface is minimal; it just allows you to specify the STM32F103  GPIO port or memory address to be accessed, e.g.

python ftdi_py_part5.py gpiob
DP ident: ack 1, value 1BA01477h
Powerup:  ack 1, value F0000040h
AP ident: ack 1, value 14770011h
40010C00: GPIOB CRL=44488433 CRH=33333344 IDR=00007FDA ODR=00007C1A

python ftdi_py_part5.py 0
DP ident: ack 1, value 1BA01477h
Powerup:  ack 1, value F0000040h
AP ident: ack 1, value 14770011h
00000000: 20005000

If a gpio port is specified, four of its register values are printed out (CRL, CRH, IDR, ODR); the other diagnostic values can be useful if the access fails. Further help is available by setting the VERBOSE flag at the top of the part 4 file, which enables a printout of all the SWD cycles:

# Command line with invalid address
python ftdi_py_part5.py 800000 
  Rd 0 IDCODE  1BA01477 Ack 1
DP ident: ack 1, value 1BA01477h
  Wr 0 ABORT   0000001E Ack 1
  Wr 4 CTRL    50000000 Ack 1
  Rd 4 STATUS  F0000000 Ack 1
Powerup:  ack 1, value F0000000h
  Wr 8 SELECT  000000F0 Ack 1
  Rd C DRW/BD3 00000000 Ack 1
  Rd C DRW/BD3 14770011 Ack 1
AP ident: ack 1, value 14770011h
  Wr 8 SELECT  00000000 Ack 1
  Wr 0 CSW/BD0 22000012 Ack 1
  Wr 4 TAR/BD1 00800000 Ack 1
  Rd C DRW/BD3 14770011 Ack 1
  Rd C DRW/BD3 00000000 Ack 4
00800000: ?

Source code

Here is the final batch of source-code. I’ve only tested the it with STM32F1 CPUs, so if you are trying to communicate with something else, there is a strong possibility you’ll need to make some changes. Points to bear in mind:

  1. Check the the CPU supports SWD, and the connections it uses.
  2. Check for any special power-up requirements, e.g. sending the reset sequence multiple times, or setting registers to enable debugging mode
  3. Check that the SWD clock and signal lines are toggling OK.
  4. Watch out for acknowledgement values 2 and 4, indicating a problem.
  5. Once an error occurs, it will persist over successive cycles until reset by writing to the ABORT register.
  6. Good luck!
# Python FTDI SWD CPU memory read from iosoft.blog
# Compatible with Python 2.7 or 3.x
#
# v0.01 JPB 8/12/18

import sys, ftdi_py_part3 as ft, ftdi_py_part4 as swd
from ctypes import Structure, Union, c_uint

# STM32F1 address values for testing
# Address of GPIO Ports A - E on STM32F1
ports = {"GPIOA":0x40010800, "GPIOB":0x40010C00, "GPIOC":0x40011000,
         "GPIOD":0x40011400, "GPIOE":0x40011800}
# GPIO registers at offsets 0, 4, 8, 12
gpio_regs = ("CRL", "CRH", "IDR", "ODR")

# Debug Port (SWD-DP) registers
# See ARM DDI 0314H "Coresight Components Technical Reference Manual"
DPORT_IDCODE        = 0x0   # ID Code / abort
DPORT_ABORT         = 0x0
DPORT_CTRL          = 0x4   # Control / status
DPORT_STATUS        = 0x4
DPORT_SELECT        = 0x8   # Select
DPORT_RDBUFF        = 0xc   # Read buffer

# Access Port (AHB-AP) registers, high nybble is bank number
# See ARM DDI 0337E: Cortex M3 Technical Reference Manual page 11-38
APORT_CSW           = 0x0   # Control status word
APORT_TAR           = 0x4   # Transfer address
APORT_DRW           = 0xc   # Data read/write
APORT_BANK0         = 0x10  # Banked registers
APORT_BANK1         = 0x14
APORT_BANK2         = 0x18
APORT_BANK3         = 0x1c
APORT_DEBUG_ROM_ADDR= 0xf8   # Address of debug ROM
APORT_IDENT         = 0xfc   # AP identification

# DP Select Register
class DP_SELECT_REG(Structure):
    _fields_ = [("DPBANKSEL",   c_uint, 4),
                ("APBANKSEL",   c_uint, 4),
                ("Reserved",    c_uint, 16),
                ("APSEL",       c_uint, 8)]
class DP_SELECT(Union):
    _fields_ = [("reg",   DP_SELECT_REG),
                ("value", c_uint)]
dp_select = DP_SELECT()

# AHB-AP Control Status Word Register
class AP_CSW_REG(Structure):
    _fields_ = [("Size",        c_uint, 3),
                ("Res1",        c_uint, 1),
                ("AddrInc",     c_uint, 2),
                ("DbgStatus",   c_uint, 1),
                ("TransInProg", c_uint, 1),
                ("MODE",        c_uint, 4),
                ("Res2",        c_uint, 13),
                ("HProt1",      c_uint, 1),
                ("Res3",        c_uint, 3),
                ("MasterType",  c_uint, 1),
                ("Res4",        c_uint, 2)]
class AP_CSW(Union):
    _fields_ = [("reg",   AP_CSW_REG),
                ("value", c_uint)]
ap_csw = AP_CSW()

# Select AP bank, do read cycle
def ap_banked_read(d, addr):
    dp_select.reg.APBANKSEL = addr >> 4;
    swd.swd_wr(d, swd.SWD_DP, DPORT_SELECT, dp_select.value)
    swd.swd_rd(d, swd.SWD_AP, addr&0xf)
    return swd.swd_rd(d, swd.SWD_AP, addr&0xf)

# Configure AP memory accesses
def ap_config(d, inc, size):
    dp_select.reg.APBANKSEL = 0     # Zero bank
    swd.swd_wr(d, swd.SWD_DP, DPORT_SELECT, dp_select.value)
    ap_csw.reg.MasterType = 1
    ap_csw.reg.HProt1 = 1           # Enable incrementing, set access size
    ap_csw.reg.AddrInc = 1 if inc else 0
    ap_csw.reg.Size = 0 if size==8 else 1 if size==16 else 2
    return swd.swd_wr(d, swd.SWD_AP, APORT_CSW, ap_csw.value)

# Set AP memory address
def ap_addr(d, addr):
    swd.swd_wr(d, swd.SWD_AP, APORT_TAR, addr)
    swd.swd_idle_bytes(d, 2)            # Idle to avoid 'wait' response

# Do an immediate read of a 32-bit CPU memory location
def cpu_mem_read32(d, addr):
    ap_addr(d, addr)                              # Address to read
    swd.swd_rd(d, swd.SWD_AP, APORT_DRW)          # Dummy read cycle
    r = swd.swd_rd(d, swd.SWD_AP, APORT_DRW)      # Read data
    return r.data.value if r.ack.value==swd.SWD_ACK_OK else None

if __name__ == "__main__":
    mem = sys.argv[1].upper() if len(sys.argv) > 1 else None
    dev = ft.ft_open()
    if not dev:
        print("Can't open FTDI device")
        sys.exit(1)
    ft.set_bitmode(dev, 0, 2)           # Enable SPI
    ft.set_spi_clock(dev, 1000000)      # Set SPI clock
    ft.ft_write(dev, (0x80, 0, ft.OPS)) # Set outputs
    swd.swd_reset(dev)                  # Send SWD reset sequence
    resp = swd.swd_rd(dev, swd.SWD_DP, DPORT_IDCODE) # Request & response
    if resp is None:
        print("No response")
    else:
        print("DP ident: ack %u, value %08Xh" % (resp.ack.value, resp.data.value))
        swd.swd_wr(dev, swd.SWD_DP, DPORT_ABORT, 0x1e)    # Clear errors
        swd.swd_wr(dev, swd.SWD_DP, DPORT_CTRL,  0x5<<28) # Powerup request
        resp = swd.swd_rd(dev, swd.SWD_DP, DPORT_STATUS)  # Get status
        print("Powerup:  ack %u, value %08Xh" % (resp.ack.value, resp.data.value))
        resp = ap_banked_read(dev, APORT_IDENT)           # Get AP ident
        print("AP ident: ack %u, value %08Xh" % (resp.ack.value, resp.data.value))
        ap_config(dev, 1, 32);                            # Configure AP RAM accesses
        s = ""
        if mem in ports:
            addr = ports[mem]
            s = "%08X: %s" % (addr, mem)
            for reg in gpio_regs:
                val = cpu_mem_read32(dev, addr)
                s += " %s=%08X" % (reg, val)
                addr += 4
        else:
            try:
                addr = int(mem, 16)
            except:
                addr = None
            if addr is not None:
                s = "%08X:" % addr
                val = cpu_mem_read32(dev, addr)
                s += " %08X" % val if val is not None else " ?"
        print(s)   

    dev.close()

# EOF

Next steps

Whilst this code is an interesting framework for experimentation, it lacks some of the features and error-handling of a ‘real’ application, for example it’d be nice to draw an animated picture of the CPU, showing the SWD poll results graphically. That is the subject of another post.

Copyright (c) Jeremy P Bentham 2018. Please credit this blog if you are using the information or software in it.