Home Measurement: Switch to RRDtool

My mobile wifi ESP8266-sensors with DHT22 (and soon even with a barometric sensor ... ) are working very well for seeing what is going on in my flat. But MRTG has been developed for monitoring internet traffic. Thus negative values are not possible to be displayed. There is no easy solution to add that. So I needed to switch to RRDtool to fetch my data and plot the graphs. This needs a bit more thinking as you need to setup databases for each of your sensors and think about how often you want to sample the data, the allowed values, how far are averages stored, and so on.

As I still want to monitor my internet traffic as well, next to the "weather sensors" I need to setup a traffic counter manually as well. My solution looks like this:

First, I generate the databases.

Traffic:

rrdtool create /var/www/rrd/fritzbox.rrd -s 300 DS:in:DERIVE:600:0:6500000 DS:out:DERIVE:600:0:1300000 RRA:AVERAGE:0.5:1:576 RRA:AVERAGE:0.5:6:672 RRA:AVERAGE:0.5:24:732 RRA:AVERAGE:0.5:144:1460

Update 24.01.2023: The traffic is measured in Byte/s. For current internet speeds, GBit/s are no unrealistic values anymore, so at least 128.000.000MByte/s are plausible. Thus, the database needs different borders, as the above limits max out occasionally with a 100MBit/s connection already – it can transfer more than 6,5 MByte/s.

rrdtool create /var/www/rrd/fritzbox.rrd -s 300 DS:in:DERIVE:600:0:400000000 DS:out:DERIVE:600:0:400000000 RRA:AVERAGE:0.5:1:576 RRA:AVERAGE:0.5:6:672 RRA:AVERAGE:0.5:24:732 RRA:AVERAGE:0.5:144:1460

Which means: sample accepted every 5 minutes (300 seconds). The bandwidth is 50MBit down and 10 MBit up, added some Bytes for the theoretical headroom. Derive means that the difference to the last sample value should be stored; with a 0 sample for example after a reboot, traffic gets counted as 0 (instead of giving a huge spike as RRDtool assumes a wrap-around of the counter otherwise). The other values create entries for the daily, weekly, monthly and yearly averages.

For the DHT22 sensors, I create a database for each sensor in place so adding new ones or removing old ones is easy.

rrdtool create /var/www/rrd/dht22-pi.rrd -s 300 DS:temp:GAUGE:600:-40:100 DS:hum:GAUGE:600:0:100 RRA:AVERAGE:0.5:1:576 RRA:AVERAGE:0.5:6:672 RRA:AVERAGE:0.5:24:732 RRA:AVERAGE:0.5:144:1460

Now I need to adopt the scripts which read out the sensors to feed the data into their database.
Bash is not quite intuitive, unfortunately. Whitespaces around "=" can take you hours to spot as error! ;)

A few hours later, everything falls into place. This is the script which gathers the data and creates the graphics. For crontab, I needed to add it with a trailing call to "/bin/bash" in the crontab-entry as it was throwing syntax errors when I directly called it.
Source here: http://pastebin.com/hMeCzweT

The graphs are embedded in a very basic HTML webpage for now:

Source here: http://pastebin.com/Vu4m4w7K (more recent version: http://pastebin.com/hPjr4Ryq ).

Now I'm still finetuning all the scripts, graphs and websites. But it is doable, although a bit more complex, to plot the data with RRD instead of MRTG.

Short Charger Review: LiitoKala Lii-500

To save some space and because the old NiMH charger startet behaving strange I decided that its time for new charger. As I know what I'm doing I'm not afraid of a charger that handles LiIon and NiMH. If you don't know or care about the battery chemistry, as worst case a deep discharged LiIon could get charged (and that with a way too high current) and become unstable, so that this cell will become dangerous. Take care to not charge LiIons that are depleated below ~2.5V.

My choice was a LiitoKala Lii-500. It has four slots which can accept even 26650 LiIon batteries, and it supports NiMH batteries as well.

On AliExpress you can find it for even less than 20€, I bought mine here.

The device is quite a bit bigger than a two-cell-charger; still two independent chargers require more space.

The Lii-500 has another nice feature that might come in handy from time to time: It can act as power bank when batteries are inserted and no power is available. A USB plug on the back side offers 5V with up to 1A then. I didn't check it yet, but I think you need at least two inserted cells for that feature to work.

 But more technically speaking, the charger is built solid. The contacts are great and work even with longer protected 18650 cells (70mm+).
For a switching charger, the charging current is very stable. A proper CC-CV charging is applied, the charger completely stops charging when the battery has 4.22V. At lest for 24-48 hours; if you leave batteries longer than that in the charger, they get slowly charged more. After 5 days I had a battery at 4,20V while it should have dropped to 4,15V or less. So take the batteries out the chargers the day after they're full, or better even earlier. By default, the Lii-500 wants to charge with 500mA; it offers 300, 500, 700 and 1000mA as well.
Also a fast or a normal test mode is included. There the battery gets discharged and fully loaded once to measure the capacity of the cell. This take some more hours though. Discharging is done with 250mA when 300 or 500mA are chosen as charge current, and 500mA for higher charging rates. The capacity shown by the charger is very close to that I can measure with my setup.

NiMH batteries get a trickle charge. Charge termination will be triggered by dV detection and seems to work properly and stable for all the tests I did the last few weeks.

It is possible to charge LiIon and NiMH batteries at the same time (in different slots, of course ;) ). The Lii-500 detects the cell type based upon the voltage and decides from there if the NiMH or LiIon charging scheme will be applied. Below 0.6V a cell is not detected. Deeply discharged NiMH cells must be "jump started" otherwise therefore - or just don't abuse your cells this badly!

I'm very satisfied with this charger's performance and can recommend it.

Home Measurement with Raspberry Pi, ESP8266 and DHT22

I have a Raspberry Pi A+ sitting in my kitchen window for over a year already, using it as webcam (first idea was some timelapse over a year with all seasons. Too much movement in the first time, so the whole-year overview takes a little longer.) Yesterday I remembered that I also have a few DHT22 temperature and humidity sensors flying around.

So I decided to hook one up to the RasPi and see what it would spit out. Just wired it up to GND, 3.3V and to GPIO17, modified a Python script which uses the AdaFruit DHT library which I found on the net, and it spits out what I need. Here is a guide from them for a starting point.

This data needs to be visualized somehow. I remembered using MRTG 15 years ago. Good news is that MRTG is still maintained and easily available. Setting it up and adding the sensor to /etc/mrtg.cfg:

### Global Config Options
WorkDir: /var/www/mrtg
Options[_]: growright, nobanner
EnableIPv6: no
WriteExpires: Yes

######################################################################
# System: CamPI DHT22
# Description: Temperature + Humidity in Kitchen Windows
######################################################################

Target[CamPi-dht]: `/usr/local/bin/dht22.py`
Title[CamPi-dht]: Temperature and Humidity in Kitchen Window
MaxBytes[CamPi-dht]: 1000
AbsMax[CamPi-dht]: 1000
WithPeak[CamPi-dht]: dwmy
Options[CamPi-dht]: gauge, growright, nopercent, pngdate
YTicsFactor[CamPi-dht]: 0.1
Factor[CamPi-dht]: 0.1
#kMG[CamPi-dht]: ,k
YLegend[CamPi-dht]: °C / %
ShortLegend[CamPi-dht]: °C/%
Legend1[CamPi-dht]: Temperature in °C
Legend2[CamPi-dht]: Relative Humidity in %
LegendI[CamPi-dht]: Temperature °C
LegendO[CamPi-dht]: Rel. Humidity %
PageTop[CamPi-dht]: <h1>Temperature and Humidity in Kitchen Window</h1>

To get the fractions of temperature and humidity, the YTicFactor and Factor options are important.

Start the index building, let mrtg run once, and you get the first empty graphs. It takes about 10 minutes so the grpahs get filled with valid data. Remember to put mrtg to the list of cronjobs so it runs every 5 minutes.

Today I found out about ESP8266 being able to directly read out a DHT22 and offer the data as web server. I needed to flash the latest NodeMCU firmware - a simple guide is available here. You should download the latest development build of NodeMCU and flash that. Just enter the path and the name of the downloaded file under "configuration" (by hitting the gear symbol you can browse the file system for that) of the esp8266-flash.exe tool, address 0x00000 is correct. Hint: Solder that CE_PD-pin on the ESP-01-boards directly to Vcc. For flashing, you need to connect GPIO0 to GND. In all cases, an eletrolytic capacitor of 1000µF between GND and Vcc helps keeping the board stable.
This firmware offers plenty of functions which you can use with LUA, a basic programming language. ESPlorer makes it extremely easy to write and test a program. My version of init.lua looks like this (adopt SSID and PW to your Wifi).

-- setup Wifi
wifi.setmode(wifi.STATION)
wifi.setphymode(wifi.PHYMODE_N)
wifi.sta.config("SSID", "PASSWORD")
wifi.sta.connect()

--start server
srv=net.createServer(net.TCP)
srv:listen(80,function(conn)
  conn:on("receive", function(client,request)

-- read DHT22 on GPIO2
    pin = 4
    status,temp,humi = dht.readxx(pin)
    h=10*humi
    t=10*temp
    if( status == dht.OK ) then
      buf = t.."\n"..h.."\n"
    end
-- reading DHT done, now spit it out
    client:send(buf)
    client:close()
    buf = nil
    t = nil
    h = nil
    status = nil
    temp = nil
    humi = nil
    pin = nil
    collectgarbage()
  end)
end)

The temperature and humidity are now accessable via webbrowser. On the Pi, create a bash script, for example /usr/local/bin/nodemcu1.sh (make it executable via "sudo chmod a+x /usr/local/bin/nodemcu1.sh"):

wget -q -O nodemcu1.txt http://nodemcu1/
cat nodemcu1.txt
echo 0
echo NodeMCU1 Living Room
rm nodemcu1.txt

Add an entry to your /etc/mrtg.cfg like this (take the rest from the example above and adopt the device name in the brackets):

######################################################################
# System: NodeMCU1 DHT22
# Description: Temperature + Humidity in Living Room
######################################################################

Target[NodeMCU1-dht]: `/usr/local/bin/nodemcu1.sh`
Title[NodeMCU1-dht]: Temperature and Humidity in Living Room

Run the mrtg indexmaker and after that mrtg again.

Now you have a mobile temperature and humidity sensor which you can put anywhere in your Wifi range.
You only need a 3.3V power supply - either via USB and step-down, or battery and step-up. There are rumors that with connecting to pin 1 of the mcu you can enable a deep sleep mode, but as the ESP8266 is working as web server it should be always-on. My USB-Meter shows that the current drawn is always jumping around between 0-200mA. This is not bad for a web server!

Update 21.01.2016: Two more DHT-22 arrived yesterday, now I have four sensors in different rooms. I like the overview over the climate at home :)

Polarity Protection with P-MOSFET

I still need to learn a lot in electronics. Just yesterday I accidentally connected the battery wrong to my LDO XC6206 - which it seriously disliked and afterwards spat out 0.56V instead of 3.3V. No problem, I have plenty of them in the box - but this may happen more often and I don't want to always solder in a new LDO.

So I looked for ways to protect the IC from wrong polarity of the battery. Since I power a ATmega328 based 7-segment-LED watch via a LiIon battery this way which needs around 2mA current, a usual diode is no alternative. The voltage drop of ~0.7V is not usable with LiIon (3.0-4.2V) on a 3.3V LDO. A Schottky diode would be way better with only 0.3V voltage drop, but still makes quite some capacity of the LiIon unusable.

Thus I decided to use the AO3415A P-channel MOSFET. It has to be connected with Drain to the battery, Source to the Vin of the LDO. Gate connects to Ground. Since the LiIon voltage is well within the specs of the AO3415 for Vgs(max), I can even work without a Zener diode/TL431 to limit the voltage; a resistor at the Gate seems not necessary as well. This is a unusual orientation for a P-FET, the first time I soldered it in without turning it on its back ;)

The results are better than I expected. The battery has 4.171-4.172V; on Source, I measure the same value down to the millivolt.

How does this protection work? When connected correctly, the body diode is conducting from Drain to Source. The Gate gets negative compared to the Source, the FET opens fully. The resistance drops down to Rds(on) for the current voltage. The load is so little with ~2mA that there is no significant voltage drop, it should be in the nano- or micro-volt area.

The capacitance behind the FET is quite small and should pose no risk that the FET wouldn't disconnect. If the battery is connected the wrong way, no current can flow as the body diode of the P-FET will block in that direction. The XC6206 won't be destroyed anymore. With such a cheap transistor the power supply is much more reliable.

Improved Data Logger for Charger Measurements

I had some spare time during my vacation so I finally built a more solid version of my shunt and voltage divider "breakout" board which I used to measure the performance of LiIon chargers with.
I soldered everything onto a small breadboard and adopted the programming a little ...

Improved Data Logger for Charger Measurements.

As you can see, the plenty of loose cables are gone. Everything is statically soldered together so that no distortions should be possible anymore.

During the first tests, some error spots are still left. The two DuPont cables that lead from the charger to the board don't make solid contact, need to solder my own ones. And when moving the plugs in the USB ports of the computer, the voltage and current delivered are changing a little as well.

The first test: The DuPont cables for connecting
 the charger need to be replaced.

But the main goal was to minimize other distortions and that worked great. I now use a 10 mOhm resistor which can stand 1W of heat dissipation and has a <= 1% error. The ADC (ADS1115) is connected via 10 kOhm resistors to its ends, ensuring very little bias. The voltage divider consists of two 100 kOhm resistors, where the connection to the ADC has again a resistor of 1 kOhm. The OpAmp for fetching the amplified differential voltage from the shunt is removed, I'm now using the built-in PGA of the ADC to get a good resolution. 1mA current over the shunt equals 10 µV, while the resolution of the ADC is ~7µV. I should probably add some oversampling as this signal has true noise and can improve this way.

Update 22:00h: Soldered wires with better contact to the board. This helped the resistance of the whole circuit to drop by 10% to now ~90 mOhm (including the battery with its inner resistance).
Also used a USB hub so only one connection is used on the computer. Turns out that the plugs show similar behaviour on the hub when the computer is moved - you can clearly still see every movement test.
The charger seems to get even less disturbed looking at the charging curve. >1A until 4.20V are reached and then the current drops slowly, while the voltage rises up to 4.25V for a short time. This is all within LiIon specs and looks quite nice.