Last summer, I hinted that I was about to switch from the monolithic programme with all its threads to a constellation of separate processes.
The main reason was that the monolithic application needs a restart for any change in the configuration and that a crash/exception on one thread breaks everything.
True, having several separate processes impose to find a way to start them all in the first place and something else to look after them. It also uses quite a bit of memory (because of Python overhead, a tiny process is almost as memory hungry as a small one). Last but not least, the interprocess communication can be problematic.
Enters the Raspberry Pi 2
Fortunately, the Raspberry Pi Foundation released the Raspberry Pi 2 which has now a quad-core CPU, 1GB of memory (and even the Raspberry Pi 3, more recently, but I doubt it would make much difference here). At ~ 3MB of memory per process, there is plenty of available RAM! And also, starting a Python process is now almost immediate compared to the 5-10 seconds needed on the 1-B+ model.
Communication : MQTT
Thinking about it, there is not much need of communication between processes. In the majority of cases, it is all bout sending the data to the display interfaces and to a database (for the sensor part).
Note that so far, I have very little automation (besides a couple of sockets).
Anyway, the ubiquitous MQTT can solve all the communication problems... These days, it seems that there isn't a similar hub project around which isn't using MQTT either at the core or at least for plug-ins communication.
I have already detailed the way I format the topic and the payload of the messages.
Every process is now using a bootstrap library which manages the daemonisation, the MQTT communication, and logs. There are 2 types of messages : the data and metadata (starting, heartbeat, ...).
Current state
Currently the model used is the following:
Notes :
All measurements have the MQTT retain option activated to keep the last value available to a reconnecting process
Pushover is a notification system for mobiles (and desktop)
I am currently using 2 storage systems: RRD and a timeseries database (test in progress)
'display_bikes' and 'display_transport' call external webservices and/or doing web scraping of pages. Resulting data is only displayed but never stored.
Having disconnected the GPIO0 and rebooted the module, the serial console was all black. It seems that speed was now down to 9600 bauds... fair enough.
First steps with the LUA console
I don't really know LUA. Fortunately, the API is documented and there are examples available on the main website.
>print(node.info())095103111471261768512040000000
OK Version is 0.9.5 as expected. Other values are chipid(10311147), flashid(1261768), flashsize(512KB) and flashspeed(40000000). Don't know how important they are in our case.
Cool... changing firmware didn't reset the Wifi configuration! :-)
Second problem...
Depite the display of the IP address... the module didn't seem connected! :-(
Well, after a lot of investigation, I discovered that UDP usually works, ping doesn't (it might not be implemented, I don't know), and for TCP... it depends.
As first, I thought it was related to a ARP (MAC address) bug described in some forums because I too could't communicate with a machine on the same LAN. But it turned out that even with remote sites (all routed through the same gateway), sometimes it worked but most of the time it ended up in timeout. It looked like the ACK (last part of the TCP handshake) wasn't accepted by the module.
I probably tried every firmware available, including the development ones, to no avail.
Pity, this looked liked a potentially interesting product. And by the way, if official website is not informative, the Github project page has all the links.
ESPlorer
On a positive node, during this exploration of nodemcu, I discovered a Java software which facilitate the use of the module.
It is called ESPlorer and can deal with the "AT mode", Nodemcu and (in theory at least) Micropython.
Micropython
Next...
Following the instructions from the Micropython Github's page, I installed esp-open-sdk in order to compile the firmware. Unfortunately this doesn't seem to work anymore. I always ended-up with errors related to implicit declaration of function 'PIN_PULLDWN_DIS'. It seems that these macros have been removed in the Expressif SDK 1.1. Great!
Since I only wanted to give a quick try, I was trying to avoid having to waste too much time on compiling this firmware!
Fortunately, the good folks at Adafruit have a compiled version ready to download.
But, sadly enough, once configured I found the same problem I had with nodemcu... I can connect to some external website but there is still this weird issue with any local website. Worse, this makes Micropython crash!
I think MicroPython has a lot of potential but it is still very much work in progress.
Note : In the meantime, I "ordered" a WiPy during their Kickstarter fundraising. It is not based on the ESP8266 but on the CC3200 which is not in the same league in terms of performance, support and price.
Back to AT Firmware
Unable to solve the TCP connection kerfuffle, I decided to flash an AT firmware, thinking that using nodemcu may have broken something.
Version 0.9.5.2 didn't seem to recognise the Wifi network at all any more but the 0.9.5 booted:
AT+GMR00200.9.5(b1)compiled @ Dec 25 2014 21:40:28AI-THINKER Dec 25 2014
That said, once again UDP connexions were fine but TCP ones were also a bit dodgy (same as with nodemcu and micropython). They used to work fine! Same for ping which was no longer possible.
Trouble is, when looking for information, there are loads of info about "ESP8266 + Arduino board" (linked by Serial) which is not the setup used here (ESP8266 used as sole MCU programmed by the Arduino IDE tool-chain).
Also without any kind of "assistance" (keep in mind that the Arduino boards use a dedicated MCU just to help with the USB connection and the flashing process) it is not as straightforward as flashing an Arduino: you need to play with GPIO0 and Reset. That said, thanks to this I understood the concept of devkit mentionned by nodemcu. More about it later.
Back to square one
Not being able to run anything because of this strange bug, I decided to see if the module was beyond repair (cf what happend with my HM-12).
For this, I managed to track down the original firmware contained in a file called ESP8266_flasher_V00170901_00_Cloud Update Ready.zip. And... Miracle! the client/server fonctions were back...
So just to make sure, I thought:
Let's try again MicroPython.... Nope! (same behaviour as before)
Let's try again Nodemcu 0.9.6 (I'm becoming good at flashing :-) )... Nope! (idem)
In short, the only firmware which seem to work on my (all?) ESP-201 seems the one reporting "00170901". Sadly enough, this one is not massively stable nor really useful in standalone.
Next time ?
Looking for additional info, I discovered that the nodemcu's devkit exists in 2 (released) versions:
the v0.9, which is wide-ish usually blue (yellow for clones) and doesn't seem to work on Macs (I personally don't care but some people might)
the v1.0, which is narrower, usually black, called v2 by Seeedstudio and is supposed to be Mac friendly.
I don't exactly know how different they are otherwise. They both have USB connection, voltage regulator and 2 buttons to help with flashing : FLASH & RESET.
So I decided to order one v1.0 from ebay ~ US$11 (inc. P&P). Should arrive within 3-4 weeks.
In the meanting, I'll try to explore deeper the possibilities of both nodemcu and arduino on ESP8266 without using network connections.
