Triggering redraw of a Image object

Question

Hello,

 

We are developing a camera preview screen for our application over a Custom Desing STM32F769 board. We are using ExternBitmap to capture frames from camera and a timer to call ExternBitmap.reload() to achieve a smooth display of captured image. But this method is fine but seems very CPU intesive. 

So we are investigating a more efficient way to update our Image object. 

For example, if somebody suggest a method to update of ram area belongs to bitmap of Image without generating a new bitmap each time and trigger a redraw for Image object, I would be vary glad...

 

Any suggestions?

 

Rgrs.

 

Aykut

Answer 1

Hello Aykut,

using Extern Bitmap in this case is in fact not very efficient. Better would be to based on the Applet view implement your own view intended to deal with extern bitmap contents. In such case you can update and redraw only small areas of the bitmap. The usage of this Applet view is demonstrated in the example Applet.

Best regards

Paul Banach

Comments

Hello Aykut,

just as an inspiration. The thread LTDC reading from double buffer also addresses the application case of camera and update. Have you seen it? In particular the option 2 in the answer may be interesting for you.

In such case instead of using an intermediate RGBA8888 bitmap and implementatig the Applet metod OnGetBitmap, you could override and implement the method OnDraw. Please see the explication to the three different modes of the Applet found at the beginning of the Applet article:

With the Applet approach do you stiill use the DMA2D hardware to copy the RGB565 camera framebuffer to the RGBA8888 bitmap? You can combine it even with the above described approach of directly accessing the GUI framebuffer. This should be the fastest solution.

Best regards

Paul Banach

---

Hello Paul,

 

Sorry for late response. It was a busy week.

 

OnDraw()/OnUpdate() approach is the best fit for my application. I can have full fps from camera stream. We already designed a new hardware with ARGB8888 camera stream. Hope it will give me more CPU time for other tasks. 

At this point, my question is answered perfectly. 

Only problem is syncing the video stream and EW copy operation. I am having some issues because of VSYNC problems. I beleive I can solve this problem with double buffering..

 

I add my code snippets as a reference for others. 

Thnx.

 

Aykut

 

C Code (ew_bsp_camera.h)

/*
 * ew_bsp_camera.h
 *
 */
#ifndef EW_BSP_CAMERA_H_
#define EW_BSP_CAMERA_H_
#include "ewrte.h"
#include "ewgfxdriver.h"
#include "ewgfx.h"
#include "ewextgfx.h"
#include "ewgfxdefs.h"
#include "ewextpxl_RGBA8888.h"
#include "ewlocale.h"
#include <string.h>
#include "Graphics.h"
#include "Application.h"
#include "ew_bsp_graphics.h"
#include "camera.h"

typedef struct {
	XPoint Size;
	XRect lockArea;
	XRect updateArea;
	XBitmapLock* lock;
	unsigned long LastTime;
	int NeedUpdate;
} ew_bsp_camera_t;

XHandle *CameraInit(XPoint aSize);
XRect* CameraUpdate(XHandle * aCamera);
XHandle *CameraProgress(XHandle *aCamera);
XBitmap* CameraGetBitmap(XHandle * aCamera);
void CameraDraw(XHandle aCamera, GraphicsCanvas aCanvas, XRect aClip,
		XRect aDstRect, XColor aColorTL, XColor aColorTR, XColor aColorBR,
		XColor aColorBL, XBool aBlend);
void CameraDone(XHandle * aCamera);

uint32_t guiGetFramePeriod(void);

#endif /* EW_BSP_CAMERA_H_ */

C Code (ew_bsp_camera.c)

/* Include necessary header files */
#include "ew_bsp_camera.h"

volatile uint32_t gui_frame_time = 0;
volatile uint32_t gui_frame_period = 0;

XHandle *CameraInit(XPoint aSize) {
	/* Reserve memory for the new camera instance */
	ew_bsp_camera_t* camera = EwAlloc(sizeof(ew_bsp_camera_t));
	if (!camera) {
		if (camera)
			EwFree(camera);
		return 0;
	}

	/* ... and initialize it */
	camera->Size = aSize;
	camera->LastTime = 0;
	camera->NeedUpdate = 1;

	camera->updateArea.Point1.X = 0;
	camera->updateArea.Point1.Y = 0;
	camera->updateArea.Point2.X = 0;
	camera->updateArea.Point2.Y = 0;

	camera->lockArea.Point1.X = 0;
	camera->lockArea.Point1.Y = 0;
	cameraInit(CAMERA_FRAME_BUFFER, 384, 480, FRAME_BUFFER_ADDR,
	FRAME_BUFFER_WIDTH, FRAME_BUFFER_HEIGHT, 208, 0);
	camera->lockArea.Point2.X = cameraGetHeight();
	camera->lockArea.Point2.Y = cameraGetWidth();

	cameraContinuousStart(CAMERA_FRAME_BUFFER);
	return (XHandle *) camera;
}

XRect* CameraUpdate(XHandle * aCamera) {

	ew_bsp_camera_t* camera = (ew_bsp_camera_t*) aCamera;

	if (cameraIsNewFrame() && !camera->NeedUpdate) {
		camera->NeedUpdate = true;
		return &(camera->lockArea); //update
	}
	return &(camera->updateArea); // Do not update
}

void CameraDone(XHandle * aCamera) {
	ew_bsp_camera_t* camera = (ew_bsp_camera_t*) aCamera;
	if (camera) {
		EwFree(camera);
	}
	cameraSuspend();
	cameraStop();

}

void CameraDraw(XHandle aCamera, GraphicsCanvas aCanvas, XRect aClip,
		XRect aDstRect, XColor aColorTL, XColor aColorTR, XColor aColorBR,
		XColor aColorBL, XBool aBlend) {
	ew_bsp_camera_t* camera = (ew_bsp_camera_t*) aCamera;

	/* Lock the entire bitmap for write operation */

	gui_frame_period = HAL_GetTick() - gui_frame_time;
	gui_frame_time += gui_frame_period;
	{
		cameraWaitVsync();
		camera->lock = EwLockBitmap((XBitmap*) (aCanvas->Super1.bitmap), 0,
				camera->lockArea, 0, 1);
		cameraChangeDisplayFrameBufferAddress(camera->lock->Pixel1);
		void *buf = (void *) camera->lock->Pixel1;
		uint32_t line_offset = FRAME_BUFFER_WIDTH - aDstRect.Point2.Y
				+ aDstRect.Point1.Y;
		EwBspGraphicsCopy((uint32_t) buf, cameraGetFrameBuffer(), line_offset,
				0, 384, 480, DMA2D_ARGB8888, DMA2D_INPUT_RGB565);
		EwUnlockBitmap(camera->lock);
	}
	camera->NeedUpdate = false;

}

uint32_t guiGetFramePeriod(void) {
	return gui_frame_period;
}

 

Chora Code

 

class CameraAppletClass : Views::Applet
{
  $rect <460,200,600,240>
  inherited method OnDraw()
  {
    $if (!$prototyper)
    native
    {
      CameraDraw(aHandle,aCanvas,aClip,aDstRect,aColorTL,aColorTR,aColorBR,aColorBL,aBlend);
    }
    $else
    aHandle;
    aColorTL;
    aColorTR;
    aColorBR;
    aColorBL;
    aBlend;

    // If the applet appears in a separate layer - a transparent gap in the UI layer
    // is needed in order to make the separate layer visible
    aCanvas.FillRectangle( aClip, aDstRect, #00000000, #00000000, #00000000,
      #00000000, false );
    $endif

  }

  $rect <460,150,610,190>
  inherited method OnUpdate()
  {
    $if (!$prototyper)
      var rect area = <0,0,0,0>;
      native ( aHandle, area )
      {
        XRect *t = (XRect *)CameraUpdate(aHandle );
        area.Point1.X = t->Point1.X;
        area.Point1.Y = t->Point1.Y;
        area.Point2.X = t->Point2.X;
        area.Point2.Y = t->Point2.Y;
      }
      return area;
    $else
      return this.Bounds.area;
    $endif

  }

  $rect <10,60,160,100>
  inherited method OnDestroy()
  {
    // TO DO: Write your code here ... 
    native {
      CameraDone(aHandle);
    }
  }

  $rect <10,10,160,50>
  inherited method OnCreate()
  {
    var handle p=null;
    native (aSize)
    {
      p = CameraInit(aSize);
    };

    this.ProgressTimer.Period=10;
    this.ProgressTimer.Enabled=true;
    return p;
  }

  // The Done() destructor forces the deinitialization of the infrastructure inherited \
  // from the base class Views::Applet.
  $rect <160,140,380,180>
  method void Done()
  {
    // When the object is disposed - destroy the embedded application
    //
    // Destroy() will then call the OnDestroy() method.
    Destroy();
  }

  // The Init() constructor forces the initialization of the infrastructure inherited \
  // from the Views::Applet class.
  $rect <160,100,380,140>
  method void Init( arg uint32 aArg )
  {
    // After the object has been created - initialize the external 'C' application.
    //
    // Create() will then call the OnCreate() method.
    Create();
  }
}

$rect <730,20,930,60>
inline Inline
{
  #include "ew_bsp_camera_ext.h"

}

 

---

Thank you Aykut for sharing your findings and the code snippets.

I hope we can provide one day a camera example for STM32 boards...

---

Would this example work on a Raspberry Pi / Raspberry Pi camera? I'm trying to build a basic camera/video app with preview screen.

FYI I've got a good feel for EW Studio, but am relatively new to programming. Any help is much appreciated!!

 

I tried to add this to my project and compiling returned this error:

Compiling ../GeneratedCode/Application.c
../GeneratedCode/Application.c: In function ‘ApplicationCameraAppletClass_OnDraw’:
../GeneratedCode/Application.c:2854:3: warning: implicit declaration of function ‘CameraDraw’ [-Wimplicit-function-declaration]
   CameraDraw(aHandle,aCanvas,aClip,aDstRect,aColorTL,aColorTR,aColorBR,aColorBL,aBlend);
   ^~~~~~~~~~
../GeneratedCode/Application.c: In function ‘ApplicationCameraAppletClass_OnUpdate’:
../GeneratedCode/Application.c:2881:25: warning: implicit declaration of function ‘CameraUpdate’ [-Wimplicit-function-declaration]
     XRect *t = (XRect *)CameraUpdate(aHandle );
                         ^~~~~~~~~~~~
../GeneratedCode/Application.c: In function ‘ApplicationCameraAppletClass_OnDestroy’:
../GeneratedCode/Application.c:2901:3: warning: implicit declaration of function ‘CameraDone’ [-Wimplicit-function-declaration]
   CameraDone(aHandle);
   ^~~~~~~~~~
../GeneratedCode/Application.c: In function ‘ApplicationCameraAppletClass_OnCreate’:
../GeneratedCode/Application.c:2926:7: warning: implicit declaration of function ‘CameraInit’ [-Wimplicit-function-declaration]
   p = CameraInit(aSize);
       ^~~~~~~~~~
../GeneratedCode/Application.c: At top level:
../GeneratedCode/Application.c:2975:31: fatal error: ew_bsp_camera_ext.h: No such file or directory

 

---

Hello,

in principle you can use the Views::Applet infrastucture on all targets. It is an interface to an external application, but the implementation is always hardware specific.

This means, the above source code is dependent to the underlying hardware (in this case STM32Fx) - so this cannot be ported to an Raspberry Pi.

In case of Raspberry Pi, the integration of a camera has to be done by using a suitable Linux camera driver.

Best regards,

Manfred

Answer 2

Hello Aykut,

an other user did almost the same as you had asked a few months ago.

Here you can find the thread how this user implemented the camera view pretty well in Embedded Wizard.

Kind regards

Tim