How can I extract an image drawn into the GUI via a touch screen?

Question

Hi, I have a simple GUI with a window in the center within which a user can draw or write an object and I would like to take this object and process it. I am using the RT 1060 with a touch screen.

 

 

Do I need to use the LCD frame buffer, as suggested in https://ask.embedded-wizard.de/3220/screenshot-function and select the part I want pixel by pixel (the drawing area is static) or is there a more elegant solution?

Thanks,

David Piskula

Answer 1

Hello David,

in principle there are two reasonable approaches. 

Approach 1:

1. In your GUI component add a new Image view. Arrange it to cover the area where you want the user to draw the figures.

2. Add to the component an object of the Graphics::Canvas class. You do this by drag&drop the corresponding class from the Browser view to Composer window as shown in the screen shot below. See also the documentation Create new embedded object.

3. In Inspector window configure the Canvas object's property FrameSize to a value corresponding to the size of the drawing area in pixel.

4. Assign the canvas object to the property Bitmap of the Image view (added in step 1 above). Accordingly, the Image view will show what evet the Canvas object stores actually. Canvas object can be seen as image pixel storage (a bitmap).

5. Add a new Simple Touch Handler and arrange it to cover the Image View (from step 1).

6. Add a new Slot method.

7. Connect the slot method to the property OnDrag of the Simple Touch Handler. Accordingly, every time the user touches and drags over the area, the Slot method is invoked.

8. Open the Slot method and implement following code:

var color clr = #FF0000FF;

// Draw a line segment between the latest and preceding position
Canvas.DrawLine( Image.Bounds.orect, SimpleTouchHandler.CurrentPos, SimpleTouchHandler.CurrentPos + SimpleTouchHandler.Offset, clr, clr, true );

// Trigger the associated views to update their aspect
notifyobservers Canvas;

9. Additionally you would need to override the inherited method Init.

10. Open the overriden method and implement following code:

// Clear the content of the Canvas bitmap. (Fill it with transparent color)
Canvas.FillRectangle( Image.Bounds.orect, Image.Bounds.orect, #00000000, #00000000, #00000000, #00000000, false );

So far the approach 1. When you implement it, you will see that it draws very thin line segments. I don't know whether it is what you expect.

Approach 2 works with vector graphic instead of drawing a bitmap. Following are steps for approach 2:

1. In your GUI component add a new Stroked Path view. Arrange it to cover the area where you want the user to draw the figures.

2. Modify the property Witdh of the Stroked Path view to the desired thickness of the path in pixel. Modify the property Color to the desired color value.

3. Add a new Path Data object.

4. Assign the Path data object to the propertxy Path of the Stroked Path view (from step 1).

5. Add a new Simple Touch Handler and arrange it to cover the Stroked Path View (from step 1).

6. Add two (2x) new Slot methods.

7. Connect the first slot method to the property OnPress and the second Slot method to the property OnDrag of the Simple Touch Handler. Accordingly, every time the user touches the area, the first method is invoked. When the user drags the finger, the second method is invoked.

8. Open the first slot method (associated to the property OnPress) and implement following code:

var point pos = SimpleTouchHandler.CurrentPos - StrokePath.Bounds.origin;

// Here you determine the max. number of segments the path may store
Path.InitSubPath( 0, 1000 );
Path.Begin( 0, pos.x, pos.y );

9. Now open and edit the second slot method (associated to the property OnDrag). Implement following code:

var point pos = SimpleTouchHandler.CurrentPos - StrokePath.Bounds.origin;

// Add a new segment to the path data object
Path.AddLine( 0, pos.x, pos.y );

That's all so far.

Does it help you?

Best regards

Paul Banach

Comments

Hello Paul,

Thank you for your answer. I am actually using the first approach already. What I need help with is extracting the drawn image out of the gui and into my application, so that I can process it. I didn't notice it was stored inside a Bitmap resource and I handled the image view wrong - perhaps I can access the current image through that?

Best regards

David Piskula

---

Hello David,

it depends on what you want to do with the image. One possibility is, you use the Canvas object as any other bitmap resource. For example, you can assign it to an Image view, etc. and so show it.

If you want to evaluate the raw pixel data stored in the Canvas, you will need to do this in native C code. For this you should understand, that the Canvas object is just a wrapper around a XBitmap object managed by the Graphics Engine. The Canvas object has an intern variable called bitmap of type handle. This variable stores in fact a pointer to the corresponding XBitmap object.

Knowing this you can access the XBitmap and by using the C function EwLockBitmap() obtain a pointer to its pixel memory. Then you can evaluate the content of this memory. As mentioned above, this implementation has to be done in native C code. The low-level access to graphical objects and pointers is not possible at the Chora level. You will need to mix Chora and C together. See the statement native. For example:

var handle tmp = Canvas.bitmap;

// Pass the Chora 'handle' to C native code
native ( tmp )
{
  /* The passed handle represents a pointer to XBitmap data structure */
  XBitmap* bitmap = (XBitmap*)tmp;


  /* Now you can lock and ... */
  XBitmapLock* lock = EwLockBitmap( bitmap, ... );

  /* ... evaluate the pixel data */
  [...]

  /* DONT'T forget to unlock the bitmap again */
  EwUnlockBitmap( lock );
}

The evaluation of pixel data depends on the color format how the pixel are stored. Please see the section XBitmapLock for details and examples of how to access the diverse color formats.

Please note: since you implement C code, the corresponding code can't be executed in the Prototyper window. You will need to test the functionality in the target system. If it is essential for you to run the code also during the prototyping, you can implement the C code in an Intrinsic module. The intrinsic module is in fact an ordinary Windows DLL. Embedded Wizard loads all found intrinsic modules automatically and adds their functionality to Chora allowing you to execute this. Unfortunately till now we have no documentation prepared demonstrating how to develop own intrinsic modules. If you are interesting in this, you will need to study the examples Applet or ExternBitmap. Both contain Visual Studio 2012 projects to create an Intrinsic module. You will of course need to replace the example's implementation with the code to lock and evaluate the bitmap pixel data.

Best regards

Paul Banach