Faster, optimized ESP-IDF fork + PSRAM Issues

Side note:

I am writing from the 35th Chaos Communication Congress, in Leipzig, Germany. If anyone is here and wants to chat about low.js, neonious one or JavaScript, Node.JS, ESP32 or microcontrollers in general, please get in contact.


During the development of low.js, a Node.JS port for ESP32 boards, we had a few challenges to overcome. With this blog post, I would like to give back to the ESP-IDF community with two things:

1. ESP-IDF modified to use dlmalloc

The default memory allocator in ESP-IDF is self made by Espressif (at least so it seems). It is not very fast, and becomes very slow when memory gets fragmented. This problem becomes evident when using SPI RAM.

esp-idf-dlmalloc is a fork of ESP-IDF which was modified to use dlmalloc, an industry standard memory allocator. It is almost twice as fast as the default memory allocator, and does not slow down notably with fragmented memory.

The fork has its own GitHub repository here: https://github.com/neonious/esp-idf-dlmalloc

Hopefully Espressif is interested to switch the memory allocator in the default branch of ESP-IDF too (and hey, Espressif, while you are here, maybe you want to take a look at low.js, might be interesting to offically back, too).

2. Still cache issues with PSRAM

This is not really a gift to the community, but might become one when our report helps fix this problem:

We stumbled upon the fact that cache issue with PSRAM still exist, even in the newest development environment. This can produce random crashes, even if the code is 100 % valid.

You can find the project showcasing the problem here: https://github.com/neonious/memcrash-esp32


Thank you for reading!

Our advertisment block:

  • Please take a look at www.lowjs.org for Node.JS on ESP32: scalable, feature rich programming of microcontrollers done easily
  • Please take a look at www.neonious.com for a great microcontroller board with Ethernet and Wifi wifi an integrated IDE + debugger on board and more!
  • Please take a look at our blog www.neonious-basics.com for interesting news and blog posts which educate you about electronics

6 Replies to “Faster, optimized ESP-IDF fork + PSRAM Issues”

    1. I do not know. Not sure if the pattern we used to trigger the problem is the only pattern which does not work. No idea why this actually happens, only that it does.

      Opened a issue in the ESP-IDF tracker, hopefully they will check futher ASAP.

      1. a high probability is that SPI RAM, interference (<- word key) external cause random values of "reads" or "records" leading to the suspension of the application.

        1. Good idea, but most probably not the problem, as exactly the first 2 MB work with the one CPU, the second 2 MB work with the second CPU. Also, switching to single CPU mode fixes the problem. Seems like a ESP32 internal problem to me.

    1. Thank’s for the link. The problems described there have to do with massive memory and virtual memory in general, things the ESP32 do not have. OK, PSRAM is cached, similar to virtual memory, but still dlmalloc is a massive improvement to the current situation.

Leave a Reply

Your email address will not be published. Required fields are marked *