Real-time processing in Matlab

Original blog illustration by Cali Rossi

Original blog illustration by Benedicte Rossi

Matlab is not traditionally used to do real-time analysis. Usually one goes to Labview when it comes to this. But still Matlab and Labview are both extending their capabilities toward each other market. Advanced analysis can be done in Labview nowadays and more and more data acquisition can be done using Matlab.

I am going to present some ideas to do a form of realtime processing in Matlab. Before going into details, keep in mind that pure real-time analysis is not really possible without appropriate hardware. In short, you need to have a buffer somewhere. But if you are not too demanding, you can do a lot of things.

I am going to play with the one thing most people have : a webcam.

Most webcams don’t guarantee a fixed frame rate. As long as you get a sense of motion, you are happy. But I feel it is a good example as you should be able to run it on your laptop.

But let’s start. Here is the code :

function realVideo()

% Define frame rate
NumberFrameDisplayPerSecond=10;

% Open figure
hFigure=figure(1);

% Set-up webcam video input
try
   % For windows
   vid = videoinput('winvideo', 1);
catch
   try
      % For macs.
      vid = videoinput('macvideo', 1);
   catch
      errordlg('No webcam available');
   end
end

% Set parameters for video
% Acquire only one frame each time
set(vid,'FramesPerTrigger',1);
% Go on forever until stopped
set(vid,'TriggerRepeat',Inf);
% Get a grayscale image
set(vid,'ReturnedColorSpace','grayscale');
triggerconfig(vid, 'Manual');

% set up timer object
TimerData=timer('TimerFcn', [email protected],vid},'Period',1/NumberFrameDisplayPerSecond,'ExecutionMode','fixedRate','BusyMode','drop');

% Start video and timer object
start(vid);
start(TimerData);

% We go on until the figure is closed
uiwait(hFigure);

% Clean up everything
stop(TimerData);
delete(TimerData);
stop(vid);
delete(vid);
% clear persistent variables
clear functions;

% This function is called by the timer to display one frame of the figure

function FrameRateDisplay(obj, event,vid)
persistent IM;
persistent handlesRaw;
persistent handlesPlot;
trigger(vid);
IM=getdata(vid,1,'uint8');

if isempty(handlesRaw)
   % if first execution, we create the figure objects
   subplot(2,1,1);
   handlesRaw=imagesc(IM);
   title('CurrentImage');

   % Plot first value
   Values=mean(IM(:));
   subplot(2,1,2);
   handlesPlot=plot(Values);
   title('Average of Frame');
   xlabel('Frame number');
   ylabel('Average value (au)');
else
   % We only update what is needed
   set(handlesRaw,'CData',IM);
   Value=mean(IM(:));
   OldValues=get(handlesPlot,'YData');
   set(handlesPlot,'YData',[OldValues Value]);
end

There are two functions. realVideo is the function you should run and FrameRateDisplay is a function used by a timer object to generate a sense of fixed frame rate. You will get to understand why “a sense”.

To run this example you will need to have the Image Acquisition Toolbox.

This code is, in some ways, in two parts. The first part is interacting with your webcam. It acquires a picture every time ‘trigger(vid);’ is sent.

The second part is a timer object that launch ‘FrameRateDisplay’ at a fixed interval (in this example every 0.1 seconds). I use timer as most computers don’t have a real and regular hardware trigger available. Timer rely on CPU cycles availability to launch a function at regular interval so it is not guarantee but should be quite good. Within FrameRateDisplay, the average of the picture is calculated and added to a plot.

When trying to develop real-time analysis, you should make sure that your analysis (here FrameRateDisplay) is being calculated in less time than the period of your data acquisition. On my machine, this is easily the case.
To achieve that, you can follow my optimization guidelines.

In this example, please note how I used persistent variables to limit memory allocation. I also minimize all my calls to graphics by  only updating the data, not the figure or the axes. xlabel or title are very slow functions for instance, so you should call them only once. Another good idea is to limit the size of all variables (please note the usage of uint8).

In any case, this is how it should look like :

 

 

This entry was posted in Intermediate. Bookmark the permalink.

37 Responses to Real-time processing in Matlab

  1. Fernando says:

    First of all, let me just congratulate you for this work, its pretty good.
    However i’m still getting one error after closing the image:

    Error while evaluating TimerFcn for timer ‘timer-1’

    Invalid or deleted object.

    Error using uiwait (line 49)
    Input argument must be of type figure

    Error in realVideo (line 39)
    uiwait(hFigure);

    Any idea how to solve this?

    • Aditya says:

      Just wanted to post this if anyone else faced a similar problem.

      Instead of using uiwait(hFigure), you can use ishandle(1) to check if the figure window is still open.

      % We go on until the figure is closed
      %uiwait(hFigure);

      if ~ishandle(figNum)
      % Clean up everything
      stop(TimerData);
      delete(TimerData);
      stop(vid);
      delete(vid);
      % clear persistent variables
      clear functions;
      end

  2. Jordi says:

    Thanks for your invaluable help.

  3. pantso says:

    hi there,

    is there a way of having a real time manipulation of wav files through only a matlab gui?
    i.e. having a gui that loads wavefiles and changing the speed (or even the volume) during the playback? something like a fader etc..
    Could you point us towards bibliography that deals with this kind of issues?
    Thank you in advance 😉

    • Jerome says:

      I suppose this should be possible, although I have never done it so I won’t be of any help here.
      I suggest you check the file exchange on Mathworks.

  4. Pate says:

    hi there,

    i wanted to have rgb values(actual image i.e without imagesc()) to be updated of each frame, and that must be in real time,and those rgb values must be plotted in real time as well

    if there are any suggestion i’m glad to get those
    thank you

  5. Ela says:

    Now i kindly know about digital image processing finalyear projects in Matlab

  6. Manoj says:

    So is there a way to use saved video as the camera input?

  7. Mdali says:

    Thanks Jerome. Excellent work!
    But would you please explain how the timer part (TimerData=timer…) works with FrameRateDisplay function to get the image data?

    • Jerome says:

      I want to make a post about timer object at some point. Timer objects execute a function (here FrameRateDisplay) at regular intervals. You set the parameters of the timer object, give it a function handle and run it. It will then do its job in the background until is stopped. Within FrameRateDisplay, I have the code to get the image and display.

  8. Jan says:

    Hello. Your website is great. I learn so much. Thank you!
    For this Real-time processing example, is there a way to limit the size boundary of the input? I’m thinking about drawing a bounding box and only processing the input from inside the box instead of the whole big image captured by the webcam… but I’m not so sure how to implement it, or if there’s actually a better way to do so.

    • Jerome says:

      Hi Jan,

      I think you need to adjust the ROIPosition parameters of the video object as explained here :
      http://www.mathworks.com/help/imaq/roiposition.html

      • Jan says:

        Hello. Thank you for your fast reply. I haven’t had a chance to say thanks to you until now. I try ROIposition and it is exactly what I need 😀 By the way, I just finish reading every posts on your blog and trying different tips on Matlab. It took me about 6-7 hrs (I’m a a slow reader), but totally worth the time. Now, I feel that using Matlab is fun and somewhat exciting. So, thank you very much.

  9. Good work. Many Thanks. I myself did every thing the same as you but with the difference of not using timers and trying to emulate its periodic invokes with a loop and some tic/toc and difference of remaining period times to pause. That worked well but using timer seems more clear.

    Of course using the timer has the over head of function calls. Although, the persistent tricks reduces the overhead.

  10. jasmine says:

    its working.. can you please suggest me the solution to monitor the video in my laptop when any other camera is placed at remote location?

  11. Gorish Aggarwal says:

    Hello, awesome work!
    Can you please tell me if it is possible to save the values of the graph in a Matlab data file along with some kind of timestamp to be used later on.
    Thanks in advance.

  12. ck says:

    Hi. Do you know if data can be monitored in realtime with figure inlays in GUIs ? Are they slow? Recommended ? Did you try that?

  13. ck says:

    Right, it is! I just wanted to know if you ever tried acquisition from multiple channels and plotting the data acquired in real time. How many channels and how fast sampling do you think it will be okay before Matlab struggles to keep up.

  14. Kanisto says:

    hi… can anyone help me to interface my smartphone camera with matlab for real
    time process?

  15. carlogarro says:

    Hello, i am trying to use your code, adding in a third plot the optical flow of the main image. Can you help about how to do it?
    I have already all the configurations of the optical flow.

    • carlogarro says:

      The thing is that I am only able to add it in the first execution, I don’t really know how works the second part.

      • Jerome says:

        You need to modify both side of “if isempty(handlesRaw)”.

        If handlesRaw does not exists, it creates the plots, otherwise it just update the data in the plot.

  16. reza says:

    i have an error “No webcame available” . please help 🙂

  17. Bobby says:

    Hi, thank you for your inspiring code.
    I have a question here, is it possible to capture frames from two cameras and make them a interlaced 3D image, then output processed images as video?
    Thank you!

  18. abdul says:

    hey i want to capture a sinlge frame from live video….plzzz any one can help???

  19. ivo says:

    I have a similar Problem right now. I want to read the my data not from a webcam, but from an ElGato HD60 device.

    imaqhwinfo woun’t find the corresponding adapter/hardware. do you have an idea how to fix that?

    if no, do you have any other ideas how to get the data of a Livestream into MatLab?

  20. TomForeman says:

    There’s definately a great deal to find out about this topic.
    I love all of the points you made.

  21. Wow, amazing blog format! How long have you been running a blog for?
    you made running a blog look easy. The whole glance of
    your website is fantastic, as smartly as the content material!

Leave a Reply

Your email address will not be published. Required fields are marked *