Differences

This shows you the differences between two versions of the page.

--- en:tech:rainsensor [2022/11/10 20:06] – bullar
+++ en:tech:rainsensor [2026/08/12 13:39] (current) – claude
@@ Line 1: / Line 1: @@
-====== Rain Gauge Sensor ======
+====== Rainfall Sensor ======
+===== Motivation =====
+The lush vegetation here in Thailand is thanks to the warmth but also to the sometimes heavy rainfall. Temperatures always stay in the range of +25...35°C, so there's no real summer or winter to speak of. That's nicely visible [[https://de.climate-data.org/asien/thailand/surat-thani-province/koh-samui-710324/#climate-graph|here]]. My own temperature logging in [[https://www.home-assistant.io|Home Assistant]] also confirms the high humidity.
+{{ :media:tech:solarvilla:villa_temp_humidity.png?direct&400 |}}
+Annual rainfall can reach up to 1600 mm/m<sup>2</sup> (1mm = 1 liter). In Germany, depending on the region, it's less than half that. The following two charts make that clear:
+{{ :media:tech:solarvilla:climate_dus-usm.png?direct&800 | Climate USM/DUS}}
+However, we haven't actually been measuring the real rainfall on our property so far. Reason enough for a new project...
+===== Sensors =====
+There's a huge variety of rain sensors, but most of them only detect whether it's currently raining. That can be useful for, say, automatically extending or retracting an awning. But we want to measure the amount of rain. For that we need a sensor that captures rainfall in a defined way. [[https://de.wikipedia.org/wiki/Niederschlagsmesser|Wikipedia]] has good information on the underlying principle.
+A digital rain gauge uses a funnel to collect the rainwater and channel it onto a tipping bucket. Once one side fills up, the bucket tips, empties the full half, and the other side starts filling. Counting the number of tips gives a good measure of rainfall. Of course you need to know the funnel opening size and the capacity of the tipping bucket. You can actually build a sensor like this yourself — [[https://www.thingiverse.com/thing:3039882|here]] is a good 3D-printed example. But this type of sensor is also available ready-made as an accessory for proprietary weather stations. One vendor is [[https://www.amazon.de/TFA-Dostmann-30-3161-Regensensor-FS-NEUTR/dp/B00FYSUATO/ref=asc_df_B00FYSUATO/?tag=googshopde-21&linkCode=df0&hvadid=256305048636&hvpos=&hvnetw=g&hvrand=2090987274523674214&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9043878&hvtargid=pla-427056504859&psc=1&th=1&psc=1|TFA Dostmann]]:
+{{ :media:tech:rainsensor:rain_sensor.png?direct&200 | TFA Dostmann rain sensor}}
+The sensor is battery-powered and transmits its readings on 433MHz, so it can be placed anywhere out in the garden. To use it for our own applications, such as Home Assistant, some reverse engineering is required.
+The funnel housing comes off easily. Underneath is the tipping bucket with the electronics housing behind it. A magnet in the tipping bucket triggers a reed switch inside. The battery compartment (2xAA) is accessible from below.
+{{ :media:tech:rainsensor:rain_sensor_wippe.png?direct&200 |Mechanics}}
+Inside there are two PCBs: one with the reed contact and a COB (Chip On Board), and the second is the 433MHz transmitter.
+<WRAP group>
+<WRAP half column>
+{{ :media:tech:rainsensor:rain_sensor_pcb1.png?nolink&200|CPU PCB}}
+</WRAP>
+<WRAP half column>
+{{:media:tech:rainsensor:rain_sensor_pcb2.png?nolink&210 |Sender PCB}}
+</WRAP>
+</WRAP>
+This separation makes it easy for us to tap the protocol going to the transmitter.
+===== Analysis =====
+There's a trigger button in the battery compartment that immediately sends the current reading — handy for capturing it with a logic analyzer, since you don't have to wait for the next natural frame. The first thing that stands out is that the protocol appears to be repeated 8 times, with a pause of about 9ms in between.
+{{ :media:tech:rainsensor:rain_sensor_protocol1.png?direct&700 |entire frame}}
+Zooming in further, the [[https://www.saleae.com|SALEAE Logic Analyzer]] shows an ASK (Amplitude-Shift Keying) protocol. Between an always-equal-length high pulse of about 488µs, there are two kinds of pauses: one 2ms long, the other 4ms.
+{{ :media:tech:rainsensor:rain_sensor_protocol2.png?direct&600 |zoomed in}}
+Let's take a look at what's actually being transmitted.
+===== SDR (Software Defined Radio) =====
+An affordable way to investigate the frequency range up to about 1766 MHz is a Software Defined Radio (SDR). That's more than enough for the rain sensor's 433MHz. The hardware basis is an 'old' DVB-T stick, as long as it contains the **RTL2832U+R820T2** chipset — that one can be freely programmed. I went with the DollaTek set, available online for about €25.
+{{ :media:tech:rainsensor:sdr_set.png?direct&400 | DollaTek DVB-T set}}
+We don't need the remote or the driver CD; the antenna and the stick are enough for further analysis. The stick is automatically recognized on most PCs. All further investigation was done on macOS, but should work similarly on Windows.
+For the stick we need the right analysis software. [[https://www.passion-radio.com/store/sdr-software-10|SDR software for Windows Linux Mac OS & drivers RTL-SDR]] has a good overview. On macOS the following programs are good options:
+  - [[https://GQRX.dk|GQRX]]
+  - [[https://cubicSDR.com|cubicSDR]]
+  - [[https://www.sdrangel.org|SDRAngel]]
+  - [[https://github.com/jopohl/urh|Universal Radio Hacker (URH)]]
+They're all free and open source. For my analysis I used URH since it combines all the analysis steps in one program.
+First we look at the frequency spectrum around 433MHz to check whether the rain sensor is actually transmitting a signal. In URH we start the //Spectrum Analyzer...// function and trigger the transmit frame by pressing the button in the battery compartment. Before starting the measurement you need to select the device — in our case the //RTL-SDR// USB stick. The result indeed shows a transmit signal at around 433.95MHz.
+{{ :media:tech:rainsensor:rain_sensor_spectrum.png?direct&400 |Frequency spectrum}}
+The blue block shows the signal over time (x=frequency, y=time), with color indicating signal strength (warmer = stronger). Above that, amplitude is plotted against frequency. With this we can now record the signal for further analysis, using the //Record Signal...// function. Again select the device and set the frequencies accordingly:
+<WRAP group>
+<WRAP half column>
+{{ :media:tech:rainsensor:rain_sensor_record.png?direct&400|Recorded signal}}
+</WRAP>
+<WRAP half column>
+{{:media:tech:rainsensor:rain_sensor_record_zoom.png?direct&215 |Zoomed}}
+</WRAP>
+</WRAP>
+The result now shows the recorded signal over time. We can see the eight repeating frame sections, and, zoomed in, the pulses and pauses already described in the digital signal earlier. We save this recording somewhere we can find again.
+Now it's time to create a new project in URH so we can save the next steps too. Then we go to the //Interpretation// tab and load the recorded signal, which we can now have 'interpreted'. We set the modulation to //ASK// and the number of samples/symbol to, e.g., 500. Below, the signal is now converted into a bitstream: every high-amplitude section becomes a 1 and the pauses in between become 0:
+{{ :media:tech:rainsensor:rain_sensor_interpretation.png?direct&400 |}}
+Here too we find the pulse sequence with the two differently-timed pauses: ''100000000100000000100001000010000100001...''. To decode the signal we go to the //Analysis// tab and, under //Decoding//, define a new decoding format, e.g. named PDM:
+{{:media:tech:rainsensor:rain_sensor_decode.png?direct&400|Decode}} {{:media:tech:rainsensor:rain_sensor_analysis.png?direct&710|Anaysis}}
+To use the built-in //Morse Code// decoding option, we first invert the signal with //Invert//. Then we need to set the Morse code parameters. After inversion we have exactly one ''ZERO'' followed by either 4 or 8 ''ONES'':
+  * Maximum length of 1-sequence for Low (0) = 4
+  * Minimum length of 1-sequence for High (1) = 7
+  * Number of 0s between 1-sequences (just for encoding) = 1
+With everything set correctly, the frame decodes without errors. The 8 frames now appear in the list. Each frame consists of 36 bits whose meaning we now have to figure out empirically. For that we create a new message type //Rain_Sensor//. Now we can group the bits appropriately, and the respective value is shown immediately in the //Value// column. In the frame table you can see that the 6th transmitted frame is corrupted and the 8th wasn't fully transmitted. But since the data of a transmit sequence is always identical, a single correct 36-bit frame is enough. Online you can find some typical grouping examples to use as a starting template. Here's the result of the iterative decoding, summarized once more:
+<code>
+ Frame:
+0000 0011 1111 1111 2222 2222 2233 3333
+4567 8901 2345 6789 0123 4567 8901 2345
+|----|----|----|----|----|----|----|----|----
+ aaaa aaaa RRbs tttt tttt tttt rrrr rrrr CCCC
+ LSB - MSB      LSB ------ MSB LSB - MSB
+ a:  8 bit address id (random after battery change)
+ b:  1 bit battery state 0 == OK
+ s:  1 bit force TX switch 1 == switch pressed
+ R:  2 bit MSB of rain value
+ t: 12 bit signed temperature * 10 (eg: 23,1°C send as 231)
+ r:  8 bit LSB of rain value
+ C:  4 bit checksum of 8 nibbles send
+</code>
+With this knowledge in hand, we can now build the new receiver that can be integrated into [[https://www.home-assistant.io|Home Assistant]].
+===== Hardware =====
+To get at the rain sensor's signal we first need a suitable 433MHz receiver. There's a wide variety available online. When choosing one, make sure it's a superheterodyne receiver and not a direct-conversion (TRF) receiver. Direct-conversion receivers have too much bandwidth and too little gain. I went with the DollaTek RXB6 superheterodyne receiver:
+<WRAP group>
+<WRAP half column>
+{{ :media:tech:rainsensor:rain_sensor_receiver.png?direct&300|433MHz Receiver}}
+</WRAP>
+<WRAP half column>
+{{:media:tech:rainsensor:rain_sensor_antena.png?direct&300 |Antenna}}
+</WRAP>
+</WRAP>
+Be sure to order the matching 433MHz helical antenna along with it.
+The receiver module runs on 5V and provides a digital output that can be connected directly to an ESP module. Here's my solution on a piece of prototype board with an ESP8266 module, the RXB6 receiver, and a micro-USB connector. Since we have 5V USB sockets at every outlet in the house, we can do without an external power supply.
+{{ :media:tech:rainsensor:rain_sensor_gateway.png?direct&400 |Rain sensor gateway}}
+===== Software (ESPHome) =====
+Now for the software. For integration into Home Assistant, ESPHome and a //custom component// are again the way to go. The following C program handles the decoding and provides four sensors:
+  * Rain Sensor
+  * Rain Temperature Sensor
+  * Lowbat Rain Sensor
+  * ID Rain Sensor
+The pin the receiver output is connected to is defined via ''#define RX_PIN 3 ''. Every level change on this pin triggers an interrupt that decodes the known protocol on the fly using a state machine. The tipping-bucket pulses still need to be multiplied by 0.5, since one pulse corresponds to 0.5mm/m<sup>2</sup> of rainfall.
+++++ rf433_rain_sensor.h |
+<file C rf433_rain_sensor.h>
+include "esphome.h"
+//#define DEBUG
+//#define MSB
+#ifdef DEBUG
+#define debug_print(x, ...) Serial.print(x, ##__VA_ARGS__)
+#define debug_println(x, ...) Serial.println(x, ##__VA_ARGS__)
+#else
+#define debug_print(x, ...)
+#define debug_println(x, ...)
+#endif
+#define RX_PIN 3
+#define IS_START_PULSE(interval)   (interval >= 7500 && interval <= 10500)
+#define IS_MARK_PULSE(interval)   (interval >= 250 && interval <= 750)
+#define IS_LOW_PULSE(interval)  (interval >= 1200 && interval <= 2800)
+#define IS_HIGH_PULSE(interval)  (interval >= 3200 && interval <= 4800)
+#define IS_GLITCH(interval) (interval < 250)
+enum sampling_state {WAIT_FOR_START, DATA_MARK, DATA_BIT, DATA_RECEIVED };
+static const char* TAG = "RF433_Rain_Sensor";
+bool have_data = false;
+bool have_batt_data = false;
+bool have_id_data = false;
+bool lowbat = false;
+int Temp = 0;
+int Rain = 0;
+int Id = 0;
+// ---------------------------------------------------------------------------
+class RF433_Rain_Sensor : public PollingComponent {
+ public:
+  Sensor *temp_sensor = new Sensor();
+  Sensor *rain_sensor = new Sensor();
+  RF433_Rain_Sensor() : PollingComponent(500) { }
+  float get_setup_priority() const override { return esphome::setup_priority::HARDWARE; }
+  static void ICACHE_RAM_ATTR pin_ISR() {
+    static unsigned long last = 0;
+    static unsigned long micros_now;
+    static byte nibble[9] = {0,0,0,0,0,0,0,0,0};
+    static uint16_t pulse;
+    static byte sampling_state = WAIT_FOR_START;
+    static byte bits = 0;
+    static byte npos = 0;
+    static byte chksum = 0;
+    micros_now = micros();
+    pulse = micros_now - last;
+    // ignore glitches
+    if (IS_GLITCH(pulse))
+      pulse = 0;
+    else
+      last = micros_now;
+    // state machine for decoding on the fly
+    switch (sampling_state) {
+      case WAIT_FOR_START:
+        if (IS_START_PULSE(pulse))  {
+          sampling_state = DATA_MARK;
+          bits = 0;
+        }
+        break;
+      case DATA_MARK:
+        if (IS_MARK_PULSE(pulse))  {
+          sampling_state = DATA_BIT;
+        } else {
+          sampling_state = WAIT_FOR_START;
+        }
+        break;
+      case DATA_BIT:
+        // LSB
+        if (IS_LOW_PULSE(pulse))  {
+#ifdef MSB
+          nibble[bits/4] &= ~(1<<(3-(bits%4)));
+#else
+          nibble[bits/4] &= ~(1<<(bits%4));
+#endif
+          bits++;
+          sampling_state = DATA_MARK;
+        } else if (IS_HIGH_PULSE(pulse))  {
+#ifdef MSB
+          nibble[bits/4] |= 1<<(3-(bits%4));
+#else
+          nibble[bits/4] |= 1<<(bits%4);
+#endif
+          bits++;
+          sampling_state = DATA_MARK;
+        }  else  {
+          sampling_state = WAIT_FOR_START;
+        }
+        if (bits == 36) sampling_state = DATA_RECEIVED;
+        break;
+      case DATA_RECEIVED:
+        chksum = 0;
+        for (npos=0;npos<8;npos++) {
+            chksum += nibble[npos];
+        }
+        chksum &= 0x0F;
+        ESP_LOGD(TAG, "Frame Received: 0x%X%X%X%X%X%X%X%X%X", nibble[0],nibble[1],nibble[2],nibble[3],nibble[4],nibble[5],nibble[6],nibble[7],nibble[8]);
+        if (chksum == nibble[8]) {
+          Temp = (nibble[5]<<8) + (nibble[4]<<4) + nibble[3];
+          Rain = (((nibble[2] & 0xC0)>>2)<<8) + (nibble[7]<<4) + nibble[6];
+          have_data = true;
+          if ((nibble[2] & 0x8) > 0) {
+            lowbat = true;
+          } else {
+            lowbat = false;
+          }
+          have_batt_data = true;
+          Id = (nibble[1]<<4) + nibble[0];
+          have_id_data = true;
+        }
+        bits = 0;
+        sampling_state = WAIT_FOR_START;
+        break;
+    }
+  }
+  // -------------------------------------------------------------------------
+  void setup() override {
+    // Prepare RF input pin
+    Serial.end();  // we are using rx pin for reciever (3)
+    pinMode(RX_PIN, INPUT);
+    attachInterrupt(RX_PIN, pin_ISR, CHANGE);
+  }
+  // -------------------------------------------------------------------------
+  void update() override {
+    // This is the actual sensor reading logic.
+    if (have_data)
+    {
+      temp_sensor->publish_state(Temp/10.0);
+      rain_sensor->publish_state(Rain*0.5);
+      have_data = false;
+    }
+  }
+};
+// ---------------------------------------------------------------------------
+class Batt_RF433_Rain_Sensor : public PollingComponent, public BinarySensor {
+  public:
+    // constructor
+    Batt_RF433_Rain_Sensor() : PollingComponent(10000) {}
+    void setup() override {
+      // This will be called by App.setup()
+    }
+    void update() override {
+      if (have_batt_data) {
+        if (lowbat) {
+          publish_state(true);
+        } else {
+          publish_state(false);
+        }
+        have_batt_data = false;
+      }
+    }
+};
+// ---------------------------------------------------------------------------
+class ID_RF433_Rain_Sensor : public PollingComponent, public Sensor {
+  public:
+    // constructor
+    ID_RF433_Rain_Sensor() : PollingComponent(10000) {}
+    void setup() override {
+      // This will be called by App.setup()
+    }
+    void update() override {
+      if (have_id_data)
+      {
+        publish_state(Id);
+        have_id_data = false;
+      }
+    }
+};
+</file>
+++++
+In ESPHome we create a new device and add our own code via an //includes://. The file ''rf433_rain_sensor.h'' needs to be copied to ''/config/esphome''.
+<code yaml>
+esphome:
+  name: rain-sensor
+  includes:
+    - rf433_rain_sensor.h
+</code>
+Then we define the four sensors as shown in the following example.
+<code yaml>
+sensor:
+- platform: custom
+  lambda: |-
+    auto rf433_sensor_data = new RF433_Rain_Sensor();
+    App.register_component(rf433_sensor_data);
+    return {rf433_sensor_data->temp_sensor, rf433_sensor_data->rain_sensor};
+  sensors:
+  - name: "Rain Temperature Sensor"
+    unit_of_measurement: °C
+    accuracy_decimals: 1
+  - name: "Rain Sensor"
+    unit_of_measurement: mm
+    accuracy_decimals: 1
+- platform: custom
+  lambda: |-
+    auto id_rf433_sensor_data = new ID_RF433_Rain_Sensor();
+    App.register_component(id_rf433_sensor_data);
+    return {id_rf433_sensor_data};
+  sensors:
+  - name: "ID Rain Sensor"
+    unit_of_measurement: ""
+    accuracy_decimals: 0
+binary_sensor:
+- platform: custom
+  lambda: |-
+    auto batt_rf433_rain_sensor = new Batt_RF433_Rain_Sensor();
+    App.register_component(batt_rf433_rain_sensor);
+    return {batt_rf433_rain_sensor};
+  binary_sensors:
+  - name: "Lowbat Rain Sensor"
+</code>
+After integration, the data is now available in Home Assistant. The measured rainfall is transmitted as an accumulated value by the sensor. Whether and when the counter needs to be reset will have to be shown by long-term operation. The counter can capture a maximum (2<sup>10</sup>) of 512mm/m<sup>2</sup>. Time will tell...
+===== Results =====
+Tested indoors, transmission works without any issues. An extended outdoor test is still pending, since the device is currently in Germany.
+I'll add this part in 2023...
+=====Links=====
+  * [[https://www.home-assistant.io]] Home Assistant
+  * [[https://esphome.io]] ESPHome
+  * [[https://airbnb.com/h/villarollokohsamui]] You can also come stay with us...
+===== Donate =====
+If you'd like to support my work, feel free to buy me a cappuccino or so: .
+<html>
+<form action="https://www.paypal.com/donate" method="post" target="_top">
+<input type="hidden" name="hosted_button_id" value="49N24HL36GF9U" />
+<input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif" border="0" name="submit" title="PayPal - The safer, easier way to pay online!" alt="Donate with PayPal button" />
+<img alt="" border="0" src="https://www.paypal.com/en_US/i/scr/pixel.gif" width="1" height="1" />
+</form>
+</html>
-Coming soon…

TechWiki

User Tools

Site Tools

Differences

Page Tools