I’m working on a simple data logger application called ComPlotter that opens a serial port and reads an incoming plain text data stream, plotting one or more signals in realtime in a scrolling window. The incoming data is assumed to be in plain text with the same number of integers (currently assumed to be three) on each line, separated by spaces. The incoming text is printed in a log window and the data values from each line are parsed and plotted in the graph area.
ComPlotter is still very much a work in progress, but it’s starting to come together and I have a reasonable looking screenshot, so I thought I’d post a little bit about it. Here’s the screenshot of the application as it currently looks. The signals displayed in the plot in this screenshot are from a simple accelerometer circuit that I had connected to my PC. The three signals visible in the plot are the x, y and z axes of the accelerometer.
ComPlotter tries serial port numbers in descending order, starting with COM30. The idea of this is that a USB-to-serial adapter may be assigned a different COM port number each time it is plugged into the PC, but it is likely to have the highest numbered port each time. In my experience, this is a pretty reliable way of accessing the type of USB-to-serial adapter that I usually use to connect my dsPIC microcontroller circuits to my PC.
ComPlotter is written in Python using the wxPython GUI toolkit. Here’s the complete source code (all 130 lines of it).
# # ComPlotter.py - wxPython data logger GUI # # Written by Ted Burke # Last updated 11-5-2012 # import serial import wx import numpy import matplotlib matplotlib.use('WXAgg') from matplotlib.backends.backend_wxagg import FigureCanvasWxAgg from matplotlib.figure import Figure import matplotlib.pyplot as plt class DataLoggerWindow(wx.Frame): def __init__(self): wx.Frame.__init__(self, None, -1, "ComPlotter", (100,100), (640,480)) self.SetBackgroundColour('#ece9d8') # Flag variables self.isLogging = False # Create data buffers self.N = 100 self.n = range(self.N) self.M = 3 self.x = [] for m in range(self.M): self.x.append(0 * numpy.ones(self.N, numpy.int)) # Create plot area and axes self.fig = Figure(facecolor='#ece9d8') self.canvas = FigureCanvasWxAgg(self, -1, self.fig) self.canvas.SetPosition((0,0)) self.canvas.SetSize((640,320)) self.ax = self.fig.add_axes([0.08,0.1,0.86,0.8]) self.ax.autoscale(False) self.ax.set_xlim(0, 99) self.ax.set_ylim(-100, 1100) for m in range(self.M): self.ax.plot(self.n,self.x[m]) # Create text box for event logging self.log_text = wx.TextCtrl( self, -1, pos=(140,320), size=(465,100), style=wx.TE_MULTILINE) self.log_text.SetFont( wx.Font(12, wx.DEFAULT, wx.NORMAL, wx.NORMAL, False)) # Create timer to read incoming data and scroll plot self.timer = wx.Timer(self) self.Bind(wx.EVT_TIMER, self.GetSample, self.timer) # Create start/stop button self.start_stop_button = wx.Button( self, label="Start", pos=(25,320), size=(100,100)) self.start_stop_button.SetFont( wx.Font(14, wx.DEFAULT, wx.NORMAL, wx.NORMAL, False)) self.start_stop_button.Bind( wx.EVT_BUTTON, self.onStartStopButton) def GetSample(self, event=None): # Get a line of text from the serial port sample_string = self.ser.readline() # Add the line to the log text box self.log_text.AppendText(sample_string) # If the line is the right length, parse it if len(sample_string) == 15: sample_string = sample_string[0:-1] sample_values = sample_string.split() for m in range(self.M): # get one value from sample value = int(sample_values[m]) self.x[m][0:99] = self.x[m][1:] self.x[m][99] = value # Update plot self.ax.cla() self.ax.autoscale(False) self.ax.set_xlim(0, self.N - 1) self.ax.set_ylim(-100, 1100) for m in range(self.M): self.ax.plot(self.n, self.x[m]) self.canvas.draw() def onStartStopButton(self, event): if not self.isLogging: self.isLogging = True self.ser = serial.Serial() self.ser.baudrate = 38400 self.ser.timeout=0.25 # Try serial ports one by one starting # with COM30 and working downwards for m in range(29, 0, -1): self.ser.port = m try: # Try this port number self.ser.open() # We only get to here if port opened self.log_text.AppendText( 'Opened COM' + str(m+1) + '...\n') break except: # We end up here if this port number # failed to open pass if self.ser.isOpen(): # We successfully opened a port, so start # a timer to read incoming data self.timer.Start(100) self.start_stop_button.SetLabel("Stop") else: self.timer.Stop() self.ser.close() self.isLogging = False self.start_stop_button.SetLabel("Start") if __name__ == '__main__': app = wx.PySimpleApp() window = DataLoggerWindow() window.Show() app.MainLoop()
hello Ted , your works are good .I’m trying to use python under .NET , normally i use visual studio C# .NET .wich kind of enviroment and software are you using for the code above?
Thanks and ciao
Walter
Hi Walter,
Thanks for your comment. When I’m writing Python programs, I don’t use an IDE (integrated development environment) at all – I just write the code in a plain text editor (Notepad++ is my favourite one for Windows – it’s free and really fast and powerful) and then run it in a console window. You can use any text editor though – even Windows Notepad will be fine.
As you’re probably aware, Python is an interpreted language rather than a compiled language, so you don’t need to install a compiler. However, to run the example above, you need to have the Python interpreter installed (download that from python.org). You also need to install wxPython (free to download from wxpython.org) which is what I use to make graphical user interfaces in Python. Finally, because the above example accesses the serial port, you need to install the PyWin32 Python for Windows extensions (free to download from here). Once you’ve installed those three (Python, wxPython, PyWin32, and optionally Notepad++), you can just run the program in a console window by typing “python ComPlotter.py”.
Regards,
Ted
Hi Ted, I keep getting importerror numpy “no module named numpy” even though I have installed numpy-1.10.4 using the “python setup.py”. The “numpy-wininst.log” is created just like the pyserial which seems to Work. I am using Python 2.7. Any suggestions why I keep gettingthis error?
Hi Jan,
I’m not sure why you’re seeing that error. If you’re running Python on Windows, you could try running the binary installer for Numpy which is what I think I’ve mostly used in the past. Here’s a link to what I think is the correct exe installer on Sourceforge:
http://sourceforge.net/projects/numpy/files/NumPy/1.10.2/
The file you probably want is the one called “numpy-1.10.2-win32-superpack-python2.7.exe”
Hope that helps!
Ted
Hello Ted,
I dont know if you will see this, considering it was posted an year back! Why do we need to use a timer when we read from the serial port? I was wondering..
Hi Ravio,
In this program, the timer is used to call the GetSample method every 100ms while data is being recorded. Each time GetSample runs, it reads one line of text (x, y and z values) from the serial port and adds the new data to the graph. When the GetSample method finishes, the program goes back to doing whatever else needs to be done, such as responding to user input (e.g. the user clicking the “stop” button) or updating the window on the screen. Because the timer is set to trigger every 100ms, the program can get on with other tasks without worrying about the serial port again until the timer tells it that it’s time to read new data.
If a simple loop (such as a while loop) was used to read data repeatedly from the serial port, the program would freeze once recording starts. It would no longer respond to used input (there would be no way to click the stop button for example) and the window would probably stop updating. You could use a loop with a fixed number of iteration to record for a specific amount of samples, then stop recording, but the program would be unresponsive in the meantime.
There are ways you can achieve the same thing without using a timer, such as using multiple threads. However, the timer method is nice and simple. I decided to use a wxPython timer object, since the program is already using wxPython anyway. Lines 56-57 create the timer object and attach it to the GetSample method (which begins on line 67). The timer does not start running when it is created. The start/stop button callback method either starts the timer (line 119) or stops it (line 122) depending on whether recording is already underway.
You’ll notice that when the timer is started on line 119, an argument of “100” is specified which means that the timer will trigger every 100ms, which was frequent enough for my incoming data. However, if the data is arriving faster, you might need to make that number smaller. Even if you make it very short, my guess is that the fact that control is returned to the application each time the GetSample method completes will keep the application responsive to user input, even if the program spends most of the time in the GetSample method.
Hopefully that explains it?
Ted
Hello Ted,
I think it’s better to use another thread rather than a timer to read the serial data.
I am working on the same topic, but don’t know how to build a figure with python.Thinks for your sharing.
ouyou
Hi Ouyou,
I agree that the way the serial input is handled here is a little bit ugly, but my intention was to keep the structure as simple and understandable as possible (I didn’t quite achieve that!). This program was really a bit of a hack to display incoming data from microcontroller circuits I’m working on. Unfortunately, I never got a chance to finish it to a point that I’m happy with it, but I still hope to some time in the future because I frequently find myself needing something like a slightly more fully featured ComPlotter (but without making the code any longer).
Of course, there are existing programs I could use to plot incoming serial data in real time, but what I really want is something ultra simple, extremely easy to use, reasonably adaptable, but still only about 1 page of code. I have another console application called ComPrinter, which I use all the time to display text input from microcontrollers in the console. I keep the download link to it on my blog so that when I’m working on one of my student’s PCs I can download it with one or two clicks and immediately display data from the highest available serial port number (usually a USB-to-serial adapter will appear as a high COM port number). My intention was for ComPlotter to be the graphical equivalent of ComPrinter, but I never quite got around to making it as simple and reliable.
Specifically regarding your thread suggestion: Yes, it would arguably be a bit more efficient to move the serial reading to a background thread. However, unless ComPlotter is really struggling to keep up with the incoming data, it’s not going to make a lot of difference in practice.
All serial reading is performed by the GetSample method of the DataLoggerWindow class, which is executed by a 100ms timer (a wxPython timer). Each time the GetSample method runs, it reads just one line of text from the COM port, so if the data are arriving faster than 10 lines per second, then that would need to be changed in the code. However, if they’re arriving in a bit slower than that, the “self.ser.readline()” on line 69 will simply block the main thread for a short time while it’s waiting for the next line of data. Once it receives the line of data, the GetSample Method runs to the end then returns control to the wxWindows event loop which means that the user interface remains responsive during data recording. In the event that “self.ser.readline()” is waiting and waiting for a line that’s not coming (e.g. because the microconrtoller has stopped sending), it has a timeout of 250ms, so it won’t block the main thread for any longer than that. (Note: If the data are arriving slower than one line every 350ms, that would also required a change to the code because “self.ser.readline()” will always end up timing out. That 350ms is the 100ms timer interval plus the 250ms timeout period.)
Ultimately, my main point is that with or without a serial thread in the program, you can’t really get around the requirement that something in the main wxPython event loop must periodically retrieve recently arrived data that has been stored in a buffer and update the graph on screen to display it. As I understand it, this action must involve the main event loop and cannot simply be delegated in its entirety to a background thread. You could make the UI a tiny bit more responsive by moving the “ser.readline()” call to a background thread, but provided it has an appropriate timeout that stops it from blocking the main thread for long it’s no big deal. Something deep in the bowels of Windows (the serial port driver? Win32? I don’t really know what) is already buffering the incoming data, so all ComPrinter is doing is periodically collecting the data and displaying it.
Ted
Hello Ted,
Thanks for your reply!
I agree with your opinion to make things as simple as possible.And I am really appreciate your words “you can’t really get around the requirement that something in the main wxPython event loop must periodically retrieve recently arrived data that has been stored in a buffer and update the graph on screen to display it”.
It seems that there is no need for another thread in your project.But in my project,to display the data on a graph is just one section. I want to build a whole closed loop control system by using computer as a remote controller.I have made a wireless-to-serial circuit module to transfer data(from wireless) to computer.So I think It is really need another thread to process other things.
Another reason, need my serial data reading process code portable.I want to use this process in my future project(Instead of display data on a 2D graph,I want to use those data to show the real-time motion of my machine on a 3D browser).
And moreover,There is no need to keep a high frequency to update graph .there is a persistence of vision for human’s eye.I mean there is no need to synchronize the graph and the data as soon as possible.We can just keep a high speed to save the data, and a low speed to update the graph.
Ouyou