Wifi with ESP8266 - Part 4

A few months ago, I noticed a very strange behaviour of the ESP8266 modules with my main router.

And as a workaround, I used a second Router/Access Point while carrying on tests with more recent firmwares. So far no luck. Since the problem is present with AT, LUA & "Arduino" versions, I assume the bug is more on the SDK side.

Even the latest AT firmware based on the Espressif SDK — 1.4.0 at the time of writing — still exhibits the bug dispite the hopes given with a changelog such as "WiFi compatibility problem of special network card"!

A few people reported the problem as well, all using a Orange Livebox as the router. This message describes exactly what I noticed and I agree with the explanation.

Basically, on the LAN side, the Livebox is composed of 3 interfaces, all bridged together:

• A 4-port wired Gigabit switch
• A 2.4Ghz Wifi (with its own SSID)
• A 5Ghz Wifi (with its own SSID)

And on the other side of the router, a WAN port.

The ESP8266 modules (using the 2.4Ghz) seem to work with the following conditions:

• ESP & anything outside the local network => TCP/UDP OK
• ESP & other 2.4Ghz device: Bi-directional ==> TCP/UDP OK
• ESP & 5Ghz device: UDP from ESP to device ==> OK, Others (espcially TCP) KO
• ESP & wired device: Same as above: Only UDP in one direction

This sounds a bit like a joke... the module can talk (or can be accessed if port-forwarding is set-up) to anything in the world... but not to a local server!!!

TBC...

EDIT (04/09/2016):

I was recently contacted by someone called Chris who, while trying OTA on ESP8266 found out that newer versions of the "Arduino boards" (aka firmwares) seem to solve the problem. I tried on my side and indeed, the problem seems to disappear with version >=2.0.0 (current is 2.3.0)... (click update on the "Boards Manager"). Glad to see this bug gone. Thanks Chris for this great news.

Wifi with ESP8266 - Part 3

Having received (in less than 2 weeks in fact) my devkit ordered from ebay, I was able to carry on my investigations.

As noted before, the devkit comes with a USB connection and power regulator. Same principle as on the Arduino boards. The version bought is the newest devkit 1.0 (AKA v2 on Seeedstudio).

First try

NodeMCU 0.9.5 build 20150403  powered by Lua 5.1.4
Will run user.lc/user.lua in 1000ms
> Please run file.remove("user.lua") before first use.
Please run file.remove("user.lua") before first use.
Please run file.remove("user.lua") before first use.
Please run file.remove("user.lua") before first use.

OK... bizarre but a quick check with Google gives a workaround:

file.remove("user.lua")
node.restart()

According to one post, there are two Lua files inside a new board: init.lua and user.lua. These are used for factory testing purpose only. Now the message has become:

NodeMCU 0.9.5 build 20150403  powered by Lua 5.1.4
Will run user.lc/user.lua in 1000ms
> cannot open user.lua

And for information purpose, the content of the "init.lua" file:

> file.open("init.lua", "r")
> print(file.read())
--init.lua, something like this
print("Will run user.lc/user.lua in 1000ms")
tmr.alarm(0,1000,1,function()
  tmr.stop(0)
  local s,err
  if file.open("user.lc") then
    file.close()
    s,err = pcall(function() dofile("user.lc") end)
  else
    s,err = pcall(function() dofile("user.lua") end)
  end
  if not s then print(err) end
end)

> file.close()

So far so good...

Connecting to the Internet

Same procedure as last time:

• Setting "Wifi Station" mode => OK
• Registering SSID and password => OK
• Obtening a IP address => OK
• Connecting to an external website => variable
• Connecting from a browser to the module => KO

For some reason, there is a incompatibility between most firmwares and my Wifi router. :-(

Time to try with a different/more recent firmware.

Flashing disaster

As we say in french "Never Two Without Three". So after the HM-12 and the ESP-201, something was bound to go awry!

Using the latest version of esptool, I uploaded a new firmware... flashing was reported as successful but the module couldn't even boot! And this was the case with all firmware I tried.

As as I discovered, this is because the devkit 1.0 is based on the ESP-12E board. The "E" letter is important as it means the flash is 4 MB (or 32Mb) instead of the 512 KB (2Mb) standard model. The mechanism address the flash seems different and it is essential to use specific parameters like:

./esptool.py write_flash -fm=dio -fs=32m  0x0 nodemcu_latest.bin

I.e. using DIO for flash_mode (instead of QIO) and 32Mb for flash_size (instead of 2Mb).

Then, at least I could boot! But the filesystem (used by LUA) was destroyed and the list of files was returning strange data.

It took me long while to find a way clear it (obviously file.format() didn't work) but I, at last, managed it by running:

./esptool.py erase_flash

That said, it is still impossible to open any file from LUA. There are a few comments about this problem on the Seeedstudio page, so I am not alone. I actually wonder if the nodemcu builds are truly compatible with this new hardware!

EDIT (02/07/2015):

Many thanks to Johny Mattsson for his workaround to this issue and for taking the time to notify me. I am happy to report that the new esptool.py (on dev branch) now unlocks the filesystem. See pull-request note Workaround write-protected flash when using esptool.py to flash in DIO mode for more information.

New access point

I happened to have a spare (old) wifi router.

And with this new set-up, TCP connections were fine either as a client or as a server!!!!

How annoying (and unpractical as I don't really want to multiply routers). And also what a waste of time and energy. Since all types of firmwares (AT, nodemcu, arduino's) are plagued by this problem, it might be link to the Espressif's SDK I presume. But I still can't pinpoint what exactly is happening and why some IPs are reachable while others are not.

At least, I could play with my two modules.

The story so far...

• 2 modules 'ESP-201' and devkit 1.0 (using an ESP-12E)
• Can't use my main Wifi Router but works fine on another one (except with a very old original firmware for the ESP-201)
• LUA Filesystem broken on the devkit 1.0 (haven't retried on the ESP-201) making the use of nodemcu very unpractical

Like this blogger, who seems to have quite a lot of experience on ESP8266 modules, I am now convinced that Arduino's ecosystem is the way to go, at least for now.

TBC...

Wifi with ESP8266 - Part 2

Following my first foray into ESP8266 world, I wanted to flash new firmwares and try alternative ones...

As far as I know, the most advanced one is the Lua-based Nodemcu, so let's give it a try.

Trying Nodemcu

Flashing

The main website for nodemcu is very confusing because it succint, not technical and seems to be based on a devkit.

Fortunately, I found the following pages, which are easy to follow:

• http://www.whatimade.today/flashing-the-nodemcu-firmware-on-the-esp8266-linux-guide/
• http://www.whatimade.today/loading-the-nodemcu-firmware-on-the-esp8266-windows-guide/

So, I hooked GPIO0 to GND and off we went...

First problem...

Having disconnected the GPIO0 and rebooted the module, the serial console was all black. It seems that speed was now down to 9600 bauds... fair enough.

First steps with the LUA console

I don't really know LUA. Fortunately, the API is documented and there are examples available on the main website.

> print(node.info())
0       9       5       10311147        1261768 512     0       40000000

OK Version is 0.9.5 as expected. Other values are chipid(10311147), flashid(1261768), flashsize(512KB) and flashspeed(40000000). Don't know how important they are in our case.

> print(wifi.sta.getip())
192.168.1.20    255.255.255.0   192.168.1.1

Cool... changing firmware didn't reset the Wifi configuration! :-)

Second problem...

Depite the display of the IP address... the module didn't seem connected! :-(

Well, after a lot of investigation, I discovered that UDP usually works, ping doesn't (it might not be implemented, I don't know), and for TCP... it depends.

As first, I thought it was related to a ARP (MAC address) bug described in some forums because I too could't communicate with a machine on the same LAN. But it turned out that even with remote sites (all routed through the same gateway), sometimes it worked but most of the time it ended up in timeout. It looked like the ACK (last part of the TCP handshake) wasn't accepted by the module.

I probably tried every firmware available, including the development ones, to no avail.

Pity, this looked liked a potentially interesting product. And by the way, if official website is not informative, the Github project page has all the links.

ESPlorer

On a positive node, during this exploration of nodemcu, I discovered a Java software which facilitate the use of the module. It is called ESPlorer and can deal with the "AT mode", Nodemcu and (in theory at least) Micropython.

Micropython

Next...

Following the instructions from the Micropython Github's page, I installed esp-open-sdk in order to compile the firmware. Unfortunately this doesn't seem to work anymore. I always ended-up with errors related to implicit declaration of function 'PIN_PULLDWN_DIS'. It seems that these macros have been removed in the Expressif SDK 1.1. Great!

Since I only wanted to give a quick try, I was trying to avoid having to waste too much time on compiling this firmware! Fortunately, the good folks at Adafruit have a compiled version ready to download.

But, sadly enough, once configured I found the same problem I had with nodemcu... I can connect to some external website but there is still this weird issue with any local website. Worse, this makes Micropython crash!

I think MicroPython has a lot of potential but it is still very much work in progress.

Note : In the meantime, I "ordered" a WiPy during their Kickstarter fundraising. It is not based on the ESP8266 but on the CC3200 which is not in the same league in terms of performance, support and price.

Back to AT Firmware

Unable to solve the TCP connection kerfuffle, I decided to flash an AT firmware, thinking that using nodemcu may have broken something.

These are referenced on http://www.electrodragon.com/w/ESP8266_Firmware#AT_Firmware and http://bbs.espressif.com/viewforum.php?f=5.

Version 0.9.5.2 didn't seem to recognise the Wifi network at all any more but the 0.9.5 booted:

AT+GMR
00200.9.5(b1)
compiled @ Dec 25 2014 21:40:28
AI-THINKER Dec 25 2014

That said, once again UDP connexions were fine but TCP ones were also a bit dodgy (same as with nodemcu and micropython). They used to work fine! Same for ping which was no longer possible.

This was all very confusing...

Arduino

Starting with Arduino 1.6.4, there is support for ESP8266. And it is very easy to setup : http://makezine.com/2015/04/01/installing-building-arduino-sketch-5-microcontroller/.

Trouble is, when looking for information, there are loads of info about "ESP8266 + Arduino board" (linked by Serial) which is not the setup used here (ESP8266 used as sole MCU programmed by the Arduino IDE tool-chain).

Also without any kind of "assistance" (keep in mind that the Arduino boards use a dedicated MCU just to help with the USB connection and the flashing process) it is not as straightforward as flashing an Arduino: you need to play with GPIO0 and Reset. That said, thanks to this I understood the concept of devkit mentionned by nodemcu. More about it later.

Back to square one

Not being able to run anything because of this strange bug, I decided to see if the module was beyond repair (cf what happend with my HM-12).

For this, I managed to track down the original firmware contained in a file called ESP8266_flasher_V00170901_00_Cloud Update Ready.zip. And... Miracle! the client/server fonctions were back...

So just to make sure, I thought:

• Let's try again MicroPython.... Nope! (same behaviour as before)
• Let's try again Nodemcu 0.9.6 (I'm becoming good at flashing :-) )... Nope! (idem)

In short, the only firmware which seem to work on my (all?) ESP-201 seems the one reporting "00170901". Sadly enough, this one is not massively stable nor really useful in standalone.

Next time ?

Looking for additional info, I discovered that the nodemcu's devkit exists in 2 (released) versions:

• the v0.9, which is wide-ish usually blue (yellow for clones) and doesn't seem to work on Macs (I personally don't care but some people might)
• the v1.0, which is narrower, usually black, called v2 by Seeedstudio and is supposed to be Mac friendly.

I don't exactly know how different they are otherwise. They both have USB connection, voltage regulator and 2 buttons to help with flashing : FLASH & RESET.

So I decided to order one v1.0 from ebay ~ US$11 (inc. P&P). Should arrive within 3-4 weeks.

In the meanting, I'll try to explore deeper the possibilities of both nodemcu and arduino on ESP8266 without using network connections.

A suivre...