About behavior of surface cache and string cache

Question

Hello, EW Team:

      I'd like to know, if objects in the caches that are configured by EW_MAX_SURFACE_CACHE_SIZE and EW_MAX_STRING_CACHE_SIZE will be released to support other urgent usage, when mempool is nearly run out.

     Thank you.

 BR.

Stephen

Answer 1

Hello Stephen,

In the default configuration case following is true:

* When trying to allocate a new graphical resource (e.g. bitmap) and there is no sufficient memory available, the system will try to clear the surface and glyph caches. Since graphical resources are memory intensive this is very efficient.

* When trying to allocate memory for a non graphical object (e.g. for a new string) and there is no memory available, the system will per default not perform any clear operations.

* String cache is per default never purged when the memory is exhausted. This is not an issue since strings usually do not occupy a lot of memory.

* When the size of a cache exceeds a predetermined threshold, the oldest unused cache entries are discarded immediately.

To control the memory usage it is the possibility to configure the maximum size of a cache. For this purpose you use the macros EW_MAX_SURFACE_CACHE_SIZE and EW_MAX_STRING_CACHE_SIZE. If you want to limit or even prevent the system from caching strings and bitmaps, just set the macros to lower values or even to the value 0. In such case unused bitmaps and strings are discarded early or even immediately. See the section Optimizing the Data Memory (RAM) usage for more details.

Other, more advanced possibility is to configure the Embedded Wizard Runtime Environment to us the so-called immediate garbage collection. In such case when memory is exhausted the system will immediately trigger the garbage collection to discard unused Chora objects, strings and purge internal caches until there is sufficient memory available to satisfy the current allocation request.

Using the immediate garbage collection, however, is a sensitive aspect. Therefore it is per default not enabled. A correctly working immediate garbage collection requires reliable information about the CPU stack memory. More details can be found in the description of the function EwSetStackAddressArea()  (or EwSetStackBaseAddress() if you are working with EW 11 or older). Moreover, this feature is available for licenses including source code only.

I hope it answers your question.

Do you observe any problem with memory usage in your application?

Best regards

Paul Banach

Comments

Hi, Paul:

      Thank you for your detailed explanation.

      We are developing GUI on RT1172 with PXP platform package. To cut down cost, we try to use internal RAM of 2MB only without affecting user experience much. We manage to provide about 1.2MB for EW mempool. Based on EW studio simulation, the size is enough. I have below questions:

      1. Does the String means word resource that has been rasterized by EW studio for runtime usage?

      2.  Is EW_PRINT_MEMORY_USAGE opened in runtime and traversing all use cases needed to reveal all memory usage problem before confirming that RAM is adequate for the application?

      3. I have a mempool of 1MB in OCRAM which is slower and another of 200kB in DTCM which is faster. Does it make difference in performance as to assigning which one as primary and extra mempool in ewconfig.h?

 

Best regards.

Stephen

---

Hello Stephen,

let me address your questions:

Does the String means word resource that has been rasterized by EW studio for runtime usage?

String is a sequence of character codes terminated by a zero character. One character within the string occupies 2 bytes. For example, the string "Hello World" consists of 11+1-zero-character=12 characters. Such string content occupies thus 24 bytes. A string is not the rasterized text. This would be a bitmap.

Is EW_PRINT_MEMORY_USAGE opened in runtime and traversing all use cases needed to reveal all memory usage problem before confirming that RAM is adequate for the application? 

Yes, at the runtime EmWi tracks the memory usage. Doing this the implementation also remembers temporary peak values corresponding to maximum memory occupation. With EW_PRINT_MEMORY_USAGE this information is displayed. It is thus very reliable to estimate the max. amount of used memory. Please note, the described mechanisms have no knowledge about other memory allocations produced e.g. in your middleware or NXP drivers, etc. It can take in account memory allocations resulting from EmWi code only.

I have a mempool of 1MB in OCRAM which is slower and another of 200kB in DTCM which is faster. Does it make difference in performance as to assigning which one as primary and extra mempool in ewconfig.h?

Unfortunately I am not experienced enough to answer this question. I forwarded this Ask thread to my colleague. He'll take care of the answer.

Based on EW studio simulation, the size is enough. 

Please note that the values shown in EmWi studio are just an estimation. To get reliable values use EW_PRINT_MEMORY_USAGE at the runtime. It also makes sense to calculate with an additional reserve when configuring the memory size. This reserve is necessary to cover aspects like heap fragmentation. I would recommend to calculate with at least +10% and test this configuration in the target system whether it is sufficient.

Best regards

Paul Banach

---

Hi, Paul:

         I played my screen in a wild style, and there was always a chance that EW output an error code 41. That should be caused by what you have mentioned as heap fragmentation.

        

         Please help me check if configuration below is the best one for avoiding heap fragmentation. Or I'm glad to get any better suggestion from you.

#define EW_LAZY_LOAD_BITMAPS                              1
#define EW_LAZY_LOAD_BITMAPS_IF_ANIMATED_ONLY             0
#define EW_DISCARD_BITMAPS                                1
#define EW_DISCARD_BITMAPS_IF_ANIMATED_ONLY               0
#define EW_DISCARD_BITMAPS_IF_NOT_USED_IN_CURRENT_UPDATE  0
#define EW_DISCARD_BITMAPS_IF_NOT_USED_IN_RECENT_UPDATES  0
#define EW_CACHE_OFFSCREEN_SURFACES                       0

 

Best regards.

Stephen

---

Hello Stephen,

your settings seem to be optimized for a system with less RAM. Accordingly, bitmaps are loaded into the RAM only when they are actively displayed on the screen. If the surface cache is full, bitmaps are discarded even if they are referenced from the application. This mode increases the number of heap allocations/deallocations. Consequently, the heap fragmentation is more probable. On the other hand, this mode results in the less possible RAM usage.

From this point of view, your configuration is already the best one. Interesting factor to consider is the size of the surface cache (determined by the macro EW_MAX_SURFACE_CACHE_SIZE). What is its actual setting?

Or I'm glad to get any better suggestion from you.

It is difficult without knowing your application. The error code 41 indicates an 'out of memory' when trying to allocate video memory (possibly for a surface). Can you please try following:

1. Enlarge the heap to max. possible size on your target.

2. Run the application in the target and using EW_PRINT_MEMORY_USAGE track the max. memory occupation.

3. Adjust the size of the heap to the value estimated in step 2 + 10%. Test the application again.

4. You can repeat the step 3 with larger or smaller heap (+20, +25%, ...) to find the optimal heap size the application is running correctly.

Concerning the opened third question, I got following answer from my colleague. I hope it helps you to understand the resulting effects:

I have a mempool of 1MB in OCRAM which is slower and another of 200kB in DTCM which is faster. Does it make difference in performance as to assigning which one as primary and extra mempool in ewconfig.h?

When allocating memory, Embedded Wizard distinguishes between small and large objects. Within an application there are usually many small objects and few large objects. Small objects are e.g. Chora objects, strings, diverse temp. data structures, events, etc. The small objects are created and freed very often during the runtime of the application. Large objects are containing usually large bitmaps and these are longtime living instances unless you configure the system to not to use any caches.

The small objects being data structures are accessed exclusively by the CPU. The large objects containing bitmaps can be accessed by the GPU and the CPU depending on whether the operation can be accelerated by the GPU or it has to be performed by software drawing routines on the CPU.

When you configure two memory pools (primary and extra pool), the small object allocations will start with the primary pool (this means, from the beginning of the entire heap). Large object allocations will start in the extra pool (from the end of the entire heap). When one of the pools is exhausted the system will try to satisfy the respective allocation using the other pool. So it is possible that small objects will be allocated from the extra pool and large objects from the primary pool.

Small objects (since they don't occupy a lot memory) are less problematic when it comes to heap fragmentation. The large objects may be problematic. However, since you have configured the system with EW_DISCARD_BITMAPS, the system will discard the bitmaps (large objects) from the cache even if they are still referenced. This configuration seems to be optimal for less RAM systems. Just check the setting for the surface cache (EW_MAX_SURFACE_CACHE_SIZE). Try to set it to 0.

Nevertheless, there also other aspects to take in account. EW_DISCARD_BITMAPS has no effect on a bitmap which is actually involved in a drawing operation. Such bitmap has to remain in memory until the drawing is done. Embedded Wizard manages a drawing instruction buffer (so-called issue buffer) to optimize and eliminate drawing operations. This may result in multiple bitmaps pending for the completion of the drawing operation. These bitmaps will not be discarded unless all drawings in the issue buffer are done.

Also the system can discard only bitmaps which can be recovered again. This is true for bitmap resource. If your application is rendering dynamically contents into a bitmap (e.g. using Buffered components or Extern Bitmap object), such bitmaps can't be discarded as long as they are referenced from the application. Doing this the application were not able to restore the content of the discarded bitmaps.

I hope it helps you further to optimize your system.

Best regards

Paul Banach

---

Hi，Paul:

       Configuration for CACHE is as below:

#define EW_MAX_STRING_CACHE_SIZE          (256)
#define EW_MAX_SURFACE_CACHE_SIZE       (512*1024)
#define EW_MAX_GLYPH_SURFACE_WIDTH       38
#define EW_MAX_GLYPH_SURFACE_HEIGHT      64
#define EW_MAX_ISSUE_TASKS               12

       My screen size is 280*280 with RGB888. Its use case is background swapping intensive. When swapping, there is always a bitmap of size 280*280*3 read in, and maybe another purged out. If bitmap is compressed, extra RAM is needed to load and then extract. So according to your explanation, there might be 3 ways of reducing possibility of out of memory error caused by fragmentation in my understanding:

      1. Decrease EW_MAX_ISSUE_TASKS to reduce pending bitmaps in heap, but may worsen performance.

      2. Have large bitmap uncompressed, but may also do harm to performance.

      3. set EW_MAX_SURFACE_CACHE_SIZE to 0.

      Please help see if my opinion is correct. Thank you.

      

Best regards.

Stephen

---

Hello Stephen,

Because it is application and target specific, it is difficult to deduce what exact effect will an option have. Let me analyse the options:

1. Decrease EW_MAX_ISSUE_TASKS to reduce pending bitmaps in heap, but may worsen performance.

I think, this approach could have an effect if there is more than one bitmap pending in the issue buffer. If you just swap the bitmap in the background I don't expect any improvement here.

2. Have large bitmap uncompressed, but may also do harm to performance.

Whether it does harm the performance will depend on the target system. Uncompressed bitmap resources are read directly from the flash memory. If GPU is used, the GPU will read the memory. It depends thus on how fast the flash memory can be accessed (by the GPU (?) or by the CPU).

Anyway, this option is worth to test it.

 3. set EW_MAX_SURFACE_CACHE_SIZE to 0.

Actually you maintain 500 kB memory for the surface cache (for the loaded bitmaps). Reducing this value may provoke the bitmaps to be discarded early reducing the memory pressure. This approach is thus worth to be tested too.

My screen size is 280*280 with RGB888. Its use case is background swapping intensive. When swapping, there is always a bitmap of size 280*280*3 read in, and maybe another purged out. If bitmap is compressed, extra RAM is needed to load and then extract.

This is an interesting detail of your application. The format RGB888 exists for the frame buffer only. It is not available for bitmap resources. The background bitmaps you are swapping will thus be stored per default in the RGBA8888 format with 4 bytes per pixel.

If the background bitmap is opaque (it does not contain any transparency) it would be recommendable to configure the corresponding bitmap resources to be generated in format RGB565. Such bitmaps will occupy only 2 bytes per pixel. Can you verify the format of bitmaps used in your application? See the bitmap resource attribute Format for more details.

Concerning other configuration parameters, I don't see any reason for a memory problems. The string cache is just 256 bytes large. So if there are more strings loaded but not used, the strings are discarded. The glyph cache is configured with size 38 x 64 which results in ~2,5 kB RAM for this cache. It is also not too much. The issue buffer occupies 12 * 64 + 512 = 1280 bytes which is also not a lot.

I would verify the bitmap format configuration and try to reduce the size of the surface cache. Also possibly try to use the bitmaps directly from the flash by setting the Mode attribute of the respective bitmap resource.

Best regards

Paul Banach