PC / RPi camera display using PyQt and OpenCV

OpenCV is an incredibly powerful image-processing tool, but it can be difficult to know where to start – how do you grab an image from a camera, and display it in a user-friendly GUI? This post describes such an application, that runs unmodified on a PC or Raspberry Pi, Windows or Linux, Python 2.7 or 3.x, and PyQt v4 or v5.


On Windows, the OpenCV and PyQt5 libraries can be installed using pip:

pip install numpy opencv-python PyQt5

If pip isn’t available, you should be able to run the module from the command line by invoking Python, e.g. for Python 3:

py -3 -m pip install numpy opencv-python PyQt5

Installing on a Raspberry Pi is potentially a lot more complicated; it is generally recommended to install from source, and for opencv-python, this is a bit convoluted. Fortunately there is a simpler option, if you don’t mind using versions that are a few years old, namely to load the binary image from the standard repository, e.g.

sudo apt update
sudo apt install python3-opencv python3-pyqt5 

At the time of writing, the most recent version of Raspbian Linux is ‘buster’, and that has OpenCV 3.2, which is quite usable. The previous ‘stretch’ distribution has python-opencv version 2.4, which is a bit too old: my code isn’t compatible with it.

With regard to cameras, all the USB Webcams I’ve tried have worked fine on Windows without needing to have any extra driver software installed; they also work on the Raspberry Pi, as well as the standard Pi camera with the ribbon-cable interface.

PyQt main window

Being compatible with PyQt version 4 and 5 requires some boilerplate code to handle the way some functions have been moved between libraries:

import sys, time, threading, cv2
    from PyQt5.QtCore import Qt
    pyqt5 = True
    pyqt5 = False
if pyqt5:
    from PyQt5.QtCore import QTimer, QPoint, pyqtSignal
    from PyQt5.QtWidgets import QApplication, QMainWindow, QTextEdit, QLabel
    from PyQt5.QtWidgets import QWidget, QAction, QVBoxLayout, QHBoxLayout
    from PyQt5.QtGui import QFont, QPainter, QImage, QTextCursor
    from PyQt4.QtCore import Qt, pyqtSignal, QTimer, QPoint
    from PyQt4.QtGui import QApplication, QMainWindow, QTextEdit, QLabel
    from PyQt4.QtGui import QWidget, QAction, QVBoxLayout, QHBoxLayout
    from PyQt4.QtGui import QFont, QPainter, QImage, QTextCursor
    import Queue as Queue
    import queue as Queue

The main window is subclassed from PyQt, with a simple arrangement of a menu bar, video image, and text box:

class MyWindow(QMainWindow):
    text_update = pyqtSignal(str)

    # Create main window
    def __init__(self, parent=None):
        QMainWindow.__init__(self, parent)

        self.central = QWidget(self)
        self.textbox = QTextEdit(self.central)
        self.textbox.setMinimumSize(300, 100)
        sys.stdout = self
        print("Camera number %u" % camera_num)
        print("Image size %u x %u" % IMG_SIZE)
        if DISP_SCALE > 1:
            print("Display scale %u:1" % DISP_SCALE)

        self.vlayout = QVBoxLayout()        # Window layout
        self.displays = QHBoxLayout()
        self.disp = ImageWidget(self)    
        self.label = QLabel(self)

        self.mainMenu = self.menuBar()      # Menu bar
        exitAction = QAction('&Exit', self)
        self.fileMenu = self.mainMenu.addMenu('&File')

There is a horizontal box layout called ‘displays’, that seems to be unnecessary as it only has one display widget in it. This is intentional, since much of my OpenCV experimentation requires additional displays to show the image processing in action; this can easily be done by creating more ImageWidgets, and adding them to the ‘displays’ layout.

Similarly, there is a redundant QLabel below the displays, which isn’t currently used, but is handy for displaying static text below the images.

Text display

It is convenient to redirect the ‘print’ output to the text box, rather than appearing on the Python console. This is done using the ‘text_update’ signal that was defined above:

    # Handle sys.stdout.write: update text display
    def write(self, text):
    def flush(self):

    # Append to text display
    def append_text(self, text):
        cur = self.textbox.textCursor()     # Move cursor to end of text
        s = str(text)
        while s:
            head,sep,s = s.partition("\n")  # Split line at LF
            cur.insertText(head)            # Insert text at cursor
            if sep:                         # New line if LF
        self.textbox.setTextCursor(cur)     # Update visible cursor

The use of a signal means that print() calls can be scattered about the code, without having to worry about which thread they’re in.

Image capture

A separate thread is used to capture the camera images, and put them in a queue to be displayed. The camera may produce images faster than they can be displayed, so it is necessary to check how many images are already in the queue; if more than 1, the new image is discarded. This prevents a buildup of unwanted images.

IMG_SIZE    = 1280,720          # 640,480 or 1280,720 or 1920,1080
CAP_API     = cv2.CAP_ANY       # or cv2.CAP_DSHOW, etc...
EXPOSURE    = 0                 # Non-zero for fixed exposure

# Grab images from the camera (separate thread)
def grab_images(cam_num, queue):
    cap = cv2.VideoCapture(cam_num-1 + CAP_API)
    cap.set(cv2.CAP_PROP_FRAME_WIDTH, IMG_SIZE[0])
    cap.set(cv2.CAP_PROP_FRAME_HEIGHT, IMG_SIZE[1])
    if EXPOSURE:
        cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 0)
        cap.set(cv2.CAP_PROP_EXPOSURE, EXPOSURE)
        cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, 1)
    while capturing:
        if cap.grab():
            retval, image = cap.retrieve(0)
            if image is not None and queue.qsize() < 2:
                time.sleep(DISP_MSEC / 1000.0)
            print("Error: can't grab camera image")

The choice of image size will depend on the camera used; all cameras support VGA size (640 x 480 pixels), more modern versions the high-definition standards of 720p (1280 x 720) or 1080p (1920 x 1080).

The camera number refers to the position in the list of cameras collected by the operating system; I’ve defined the first camera as number 1, but the OpenCV call defines the first as 0, so the number has to be adjusted.

The same parameter is also used to define the capture API setting; by default this is ‘any’, which usually works well; my Windows 10 system defaults to the MSMF (Microsoft Media Foundation) backend, while the Raspberry Pi defaults to Video for Linux (V4L). Sometimes you may need to force a particular API to be used, for example, I have a Logitech C270 webcam that works fine on Windows 7, but fails on Windows 10 with an ‘MSMF grab error’. Forcing the software to use the DirectShow API (using the cv2.CAP_DSHOW option) fixes the problem.

If you want to check which backend is being used, try:

print("Backend '%s'" % cap.getBackendName())

Unfortunately this only works on the later revisions of OpenCV.

Manual exposure setting can be a bit hit-and-miss, depending on the camera and API you are using; the default is automatic operation, and setting EXPOSURE non-zero (e.g. to a value of -3) generally works, however it can be difficult to set a webcam back to automatic operation: sometimes I’ve had to use another application to do this. So it is suggested that you keep auto-exposure enabled if possible.

[Supplementary note: it seems that these parameter values aren’t standardised across the backends. For example, the CAP_PROP_AUTO_EXPOSURE value in my source code is correct for the MSMF backend; a value of 1 enables automatic exposure, 0 disables it. However, the V4L backend on the Raspberry Pi uses the opposite values: automatic is 0, and manual is 1. So it looks like my code is incorrect for Linux. I haven’t yet found any detailed documentation for this, so had to fall back on reading the source code, namely the OpenCV videoio ‘cap’ files such as cap_msmf.cpp and cap_v4l.cpp.]

Image display

The camera image is displayed in a custom widget:

# Image widget
class ImageWidget(QWidget):
    def __init__(self, parent=None):
        super(ImageWidget, self).__init__(parent)
        self.image = None

    def setImage(self, image):
        self.image = image

    def paintEvent(self, event):
        qp = QPainter()
        if self.image:
            qp.drawImage(QPoint(0, 0), self.image)

A timer event is used to trigger a scan of the image queue. This contains images in the camera format, which must be converted into the PyQt display format:

DISP_SCALE  = 2                 # Scaling factor for display image

    # Start image capture & display
    def start(self):
        self.timer = QTimer(self)           # Timer to trigger display
                    self.show_image(image_queue, self.disp, DISP_SCALE))
        self.capture_thread = threading.Thread(target=grab_images, 
                    args=(camera_num, image_queue))
        self.capture_thread.start()         # Thread to grab images

    # Fetch camera image from queue, and display it
    def show_image(self, imageq, display, scale):
        if not imageq.empty():
            image = imageq.get()
            if image is not None and len(image) > 0:
                img = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
                self.display_image(img, display, scale)

    # Display an image, reduce size if required
    def display_image(self, img, display, scale=1):
        disp_size = img.shape[1]//scale, img.shape[0]//scale
        disp_bpl = disp_size[0] * 3
        if scale > 1:
            img = cv2.resize(img, disp_size, 
        qimg = QImage(img.data, disp_size[0], disp_size[1], 
                      disp_bpl, IMG_FORMAT)

This demonstrates the power of OpenCV; with one function call we convert the image from BGR to RGB format, then another is used to resize the image using cubic interpolation. Finally a PyQt function is used to convert from OpenCV to PyQt format.

Running the application

Make sure you’re using the Python version that has the OpenCV and PyQt installed, e.g. for the Raspberry Pi:

python3 cam_display.py

There is an optional argument that can be used if there are multiple cameras; the default first camera is number 1.

On Linux, some USB Webcams cause a constant stream of JPEG format errors to be printed on the console, complaining about extraneous bytes in the data. There is some discussion online as to the cause of the error, and the cure seems to involve rebuilding the libraries from source; I’m keen to avoid that, so used the simple workaround of suppressing the errors by redirecting STDERR to null:

python3 cam_display.py 2> /dev/null

Fortunately this workaround is only needed with some USB cameras; the standard Raspberry Pi camera with the CSI ribbon-cable interface works fine.

Source code

Full source code is available here.

For a more significant OpenCV application, take a look at this post.

Copyright (c) Jeremy P Bentham 2019. Please credit this blog if you use the information or software in it.

PyQt serial terminal

A simple demonstration of threading in PyQt

I do a lot of work with serial comms; TeraTerm is an excellent serial terminal, but sometimes a customised application is required, for example when dealing with binary data.

The following blog describes a simple Windows & Linux serial terminal, that can be adapted to handle special protocols; it also demonstrates the creation of Python threads, and can serve as a basis for other multi-threaded applications.


It won’t win any awards for style, but could come in handy next time you encounter an obscure serial protocol.


My code is compatible with Python 2.7 and 3.x, PyQt v4 and v5, running on Windows or Linux. This necessitates some rather clunky inclusions at the top of the file:

    from PyQt4 import QtGui, QtCore
    from PyQt4.QtGui import QTextEdit, QWidget, QApplication, QVBoxLayout
    from PyQt5 import QtGui, QtCore
    from PyQt5.QtWidgets import QTextEdit, QWidget, QApplication, QVBoxLayout
    import Queue
    import queue as Queue

There is also an issue with the different ways Python 2 and 3 handle serial data; older versions assume that the data type is an ASCII string, while in later versions it is a binary type.

This can lead to all sorts of problems (such as unwanted exceptions) so I’ve included specific functions to ensure that all outgoing data is converted from the internal representation (which could be Unicode) to a string of bytes, and incoming data is converted from the external type (data bytes or string) to a string type:

# Convert a string to bytes (for Python 3)
def str_bytes(s):
    return s.encode('latin-1')

# Convert bytes to string (if Python 3)
def bytes_str(d):
    return d if type(d) is str else "".join([chr(b) for b in d])

You’ll need to install pySerial by the usual methods (e.g. ‘pip install’).

Using threading

The key problem with serial communications is the time it takes; the slower the data rate, the longer the delay before anything happens. Without threading (or any other mitigation scheme) the User Interface (UI) will lock up after each command, waiting for the response. At best, this makes the application appear sluggish and unresponsive; at worst, it can appear to have failed, waiting for a response that never comes.

Threading allows the UI to carry on interacting with the user, while simultaneously keeping the serial link alive. Creating a new thread is really easy in Python; you just subclass QThread, and instantiate it:

class SerialThread(QtCore.QThread):
    def __init__(self, portname, baudrate):
        self.portname, self.baudrate = portname, baudrate
class MyWidget(QWidget):
    self.serth = SerialThread(portname, baudrate)

I have chosen to supply the serial parameters (port name & baud rate) when the thread object is instantiated, so they are available when the thread is started; this is done by calling the start() method, which will call a run() method in the QThread class:

# In serial thread:
def run(self):
    print("Opening %s at %u baud" % (self.portname, self.baudrate))
# In UI thread:

The run() method needs to keep running in a perpetual loop, but it must be possible to terminate that loop when the program exits – Python won’t do it automatically. I use a global variable, that can be set false to terminate, e.g. in pseudocode:

def run(self):
    [starting: open serial port]
    while self.running:
        [check for incoming characters]
        [check for outgoing characters]
    [finished: close serial port]

When the application is closing, it terminates the thread by setting the boolean variable false, then (very importantly) waits for the thread to finish its execution:

def closeEvent(self, event):
    self.serth.running = False


The user will be entering keystrokes in the UI thread, and it is tempting to call the serial transmit function from that thread, but this isn’t a good idea; it is better to pass the keystrokes across to the serial thread for transmission, and we need a thread-safe method of doing this. That means we can’t just use a global shared string variable; Python does a lot of behind-the-scenes processing that could lead to an unpredictable result. Instead, we’ll use a First In First Out (FIFO) queue:

# In UI thread..
txq = Queue.Queue()
txq.put(s)           # Add string to queue
# In serial thread..
if not txq.empty():
    txd = txq.get()  # Get string from queue

So the serial thread polls the transmit queue for any data, outputting it to the serial port.


We could use the same technique for received data; the serial thread could add it to a queue that is polled by the UI thread, and somehow trigger a UI redraw when the new data arrives, but I prefer to use a signal; the data is attached to that signal, and is received by a UI function that has registered a connection. The signal has to be in a class definition, and must specify the type of data that will be attached:

class MyWidget(QWidget):
    text_update = QtCore.pyqtSignal(str)

The signal is connected to a function that will process the data:


So now the serial thread just has to generate a signal when new data is received:


This technique would be quite adequate, but I do like having the output from all my ‘print’ function calls redirected to the same window; it makes for cleaner error reporting when things go wrong, rather than having a separate console with error messages. This is done by redirecting stdout to my widget, and adding write() and flush() handlers:

class MyWidget(QWidget):
    text_update = QtCore.pyqtSignal(str)
    def __init__(self, *args): 
        sys.stdout = self
    def write(self, text):
    def flush(self):
    def append_text(self, text):        
        [add text to UI display]

So now, every time I make a print() call, a signal is sent to my append_text function, where the display is updated. The use of a signal means that I can still call print() from any thread, without fear of strange cross-threading problems.

Polling in serial thread

The serial thread is just polling for¬†incoming and outgoing characters, and if there are none, the processor will execute the ‘while’ loop really quickly. In the absence of any delays, it will consume a lot of CPU time just checking for things that don’t exist. This may appear harmless, but it is quite alarming for the user if the CPU fan starts spinning rapidly whenever your application is running, and a laptop user won’t be happy if you needlessly drain their battery by performing pointless tasks. So we need to add some harmless time-wasting to the polling loop, by frequently returning control to the operating system. This can be done by calling the ‘sleep’ function, but we still want the software to be responsive when some serial data actually arrives. A suitable compromise is to use a serial read-function with a timeout, so the software ‘blocks’ (i.e. stalls) until either some characters are received, or there is a timeout:

self.ser = serial.Serial(self.portname, self.baudrate, timeout=SER_TIMEOUT)
s = self.ser.read(self.ser.in_waiting or 1)

In case you are unfamiliar with the usage, the ‘or’ function returns the left-hand side if it is true (non-zero), otherwise the right-hand side. So every read attempt is at least 1 character, or more characters if they are available. If none are present, the read function waits until the timeout, so when the serial line is idle, most time will be spent waiting in the operating system.

User Interface

This has been kept as simple as possible, with just a text box as a main widget. One minor complication is that as standard, the text box (which is actually a QTextEdit control) will capture and display all keystrokes, so we need to subclass it to intercept the keys, and call a handler function that adds them to the serial transmit queue. I didn’t want to burden the text box with this functionality, so put the handler in its parent, which is the main widget:

class MyTextBox(QTextEdit):
    def __init__(self, *args): 
        QTextEdit.__init__(self, *args)

    def keyPressEvent(self, event):

The keystroke handler in the main widget gets the character from the key event, and checks for a ctrl-V ‘paste’ request; I’ve included this feature because I find it useful to cut-and-paste frequently-used serial commands from a document, rather than re-typing them every time.

# In main widget:
def keypress_handler(self, event):
    k = event.key()
    s = RETURN_CHAR if k==QtCore.Qt.Key_Return else event.text()
    if len(s)>0 and s[0]==PASTE_CHAR:
        cb = QApplication.clipboard() 

Interestingly, with PyQt5 you can cut-and-paste full 8-bit data (i.e. bytes with the high bit set), but this doesn’t seem to work in PyQt4, which only accepts the usual ASCII character set.

I haven’t included any menu options, you have to specify the COM port on the command-line using the -c option, and baud rate using -b. There is also a -x option to display incoming data in hexadecimal, for example:

  python pyqt_serialterm.py -c com2 -b 9600 -x
  python pyqt_serialterm.py -c /dev/ttyUSB0 -b 115200

On Linux, if you want non-root access to a serial port, you will generally need to add your username to the ‘dialout’ group:

sudo usermod -a -G dialout $USER

..then log out & back in.

Source code

The source code is posted here.

Copyright (c) Jeremy P Bentham 2019. Please credit this blog if you use the information or software in it.