Investigating missing weather station data with SDR
This documents my investigation into some recently missing data from our home weather station, and some lessons learned.
The Missing Data
I picked up this "MaxKon" branded weather station almost two years ago. It came with an outdoor unit that contained an anemometer, wind direction sensor, rainfall gauge, solar panel, and a pressure/temperature/humidity sensors. It also has an indoor unit that shows interior temperature, humidity, date, and time.
And while it used to show all of that information correctly, recently (as in, the past few months) it has shown nothing for the outdoor temperature and humidity values.
The date and time are incorrect due to the device being power-cycled recently during previous attempts to fix the issue
Attempted Corrections
To try and restore the data, a number of the seemingly obvious solutions were tried.
The batteries in the outdoor unit were replaced. The indoor unit was power-cycled from the wall power, and had its batteries replaced and power-cycled. The "Reset" button on the bottom of the outdoor unit was held for a few seconds and lit up the LED when released (indicating successful reset.) The body of the outdoor unit was cleaned of debris and insects. And yet none of this was able to restore the temperature or humidity data.
The outdoor station was brought inside, next to the base station, and the two were allowed to settle into the same state over a few hours. But this didn't bring the data back either.
Finding the Signal
At this point, I was curious to discover more about the failure. The weather station transmits in the ISM bands, within a range that can be received by an SDR device I have on-hand.
The specific frequency used by the weather station was easy to find because it was printed on the back of the base station.
Weather Station(reciever)
Model: FT-0835
Power: 3 x1.5V AAA battery
Frequency: 433.9MHz
AUSWAY PACIFIC PTY LTD
Preparing the Dipole
Knowing the frequency in advance also helped ensure the antenna length was ideal.
The antenna being used is a dipole, and the total antenna length is calculated with
L = 468 / f
which makes each leg one half of L.
For a frequency of 433 MHz this results in a leg length of ~16.5 cm.
Visual Confirmation
I used SDR++ in this case to search the airwaves for proof that the weather station was transmitting in the 433 MHz range.
After some configuration (offset tuning, IQ Correction, RTL AGC, and Tuner AGC) something that appeared to be the correct signal could be seen loud and clear. Right on 433.91 MHz there were small bursts of data being emitted once every 15 seconds or so. They seem to alternate one short then one long burst.
This provided good evidence that the station was transmitting the data on the frequency expected, but more work is required to parse the data within the signal.
Trying to Collect the Data
There is a piece of software built precisely for the purpose of processing data on the ISM bands, RTL-433.
Despite the name, RTL-433 isn't only for the 433 MHz frequency. The readme says it is "mainly for the 433.92 MHz, 868 MHz (SRD), 315 MHz, 345 MHz, and 915 MHz ISM bands". But in this case, it is indeed being used for the 433 MHz band and because of that we should be able to execute the program with its default parameters.
$ rtl_433
rtl_433 version unknown inputs file rtl_tcp RTL-SDR SoapySDR
Use -h for usage help and see https://triq.org/ for documentation.
Trying conf file at "rtl_433.conf"...
Trying conf file at "/home/admin/.config/rtl_433/rtl_433.conf"...
Trying conf file at "/usr/local/etc/rtl_433/rtl_433.conf"...
Trying conf file at "/etc/rtl_433/rtl_433.conf"...
Registered 145 out of 175 device decoding protocols [ 1-4 8 11-12 15-17 19-21 23 25-26 29-36 38-60 63 67-71 73-100 102-105 108-116 119 121 124-128 130-149 151-161 163-168 170-175 ]
Found Elonics E4000 tuner
Exact sample rate is: 250000.000414 Hz
Sample rate set to 250000 S/s.
Tuner gain set to Auto.
Tuned to 433.910MHz.
Allocating 15 zero-copy buffers
However, that didn't work for me. The program would remain in that state, without reporting the detection of any signals.
Trying to Reveal the Signal
Some of the documentation suggests tuning slightly off-center of the target frequency because the DC spike within the SDR dongle can cause interference in the center. This seemed like it could be a likely issue for my dongle because within SDR++ there was significant interference in the center (the "DC Spike") until the "offset tuning" option was selected within that program. There was an attempt to tune around the target frequency. Attempts were made with center frequencies of 433.92M, 433.91M, 433.9M, 433.89M, and 433.87M, but none reported any signals.
At this point, I wanted to try to clearly see what the program was seeing coming in from the SDR dongle. This could help show if the center spike was under control, and could help show how clearly the weather station's signal was showing up.
The RTL-433 program allows for files to be captured with a command like:
$ rtl_433 -f 433.8M -w file_433.8M_250k.cu8 -T 60
This produces a recording of the data, and can be uploaded to a tool like https://triq.org/pdv/ for analysis.
Note:
The "433.8M_259k" part of the filename above tells the triq.org/pdv/ tool that the sample is centered on 433.8 MHz and has a 250k sample rate. It should be updated for your situation.
This provided a way to visualize what RTL-433 was seeing.
This helped explain why the software was failing to find the transmissions. The transmissions from the station were visible to the side of the center spike, which is good. But they may be too faint, or to similar in intensity to the center spike, and are perhaps being ignored by the software.
Piping RTL-SDR into Baudline
I was also able to get a live view the SDR dongle's raw view of the of the 433 MHz band - without the tweaking and modifications performed by SDR++ - by piping data directly from RTL-SDR into baudline, a signal analysis tool.
Here's an example command to pipe the frequency of 433.87 MHz at a sample rate of 1,000,000 into baudline for viewing:
$ rtl_sdr -f 433870000 -s 1000000 - | baudline -reset -samplerate 1000000 -channels 2 -format u8 -quadrature -stdin
This live view was useful for more rapidly testing the behavior of the dongle. It helped me clearly see the benefit of tuning slightly off-center of the target frequency because the actual transmission was only visible when not centered, otherwise it was completely indiscernible from the center spike.
Updating RTL-433
Because the center spike was so significant, I was curious if RTL-433 had a method to do the same "offset tuning" that SDR++ allowed, since simply manually changing the center frequency wasn't making the signal clear enough yet.
When searching the documentation of RTL-433, it mentions some settings that can be sent to the device:
[-t <settings>] apply a list of keyword=value settings to the SDR device
e.g. for SoapySDR -t "antenna=A,bandwidth=4.5M,rfnotch_ctrl=false"
for RTL-SDR use "direct_samp[=1]", "offset_tune[=1]", "digital_agc[=1]", "biastee[=1]"
This appeared to be exactly what was needed, it should be possible to pass -t "offset_tune=1". However, when providing this parameter the software complained that it wasn't allowed for my device:
WARNING: sdr settings only available for SoapySDR devices.
This was odd because the documentation mentioned more than just SoapySDR devices, and made me suspect that I may have an older version of RTL-433 installed. I attempted to use apt install to upgrade the rtl-433 package but it maintained that the currently installed version was the latest.
The -V flag can be used to print the current version installed:
$ rtl_433 -V
rtl_433 version unknown inputs file rtl_tcp RTL-SDR SoapySDR
That was pretty odd. According to the program, the version was "unknown". Checking the man page for the program indicated it might have actually been a version from 2019, a few years old.
To get the latest version, I decided to clone the github repo and build it from source according to the instructions. This worked painlessly and the local software finally reported a sensible version:
$ rtl_433 -V
rtl_433 version 22.11-246-g2a7fe211 branch master at 202311011021 inputs file rtl_tcp RTL-SDR SoapySDR
Finally Capturing the Transmission
This updated version of RTL-433 permitted sending settings to the SDR device with the -t flag, without complaining that the device wasn't a SoapySDR device.
However, this didn't immediately fix everything.
Trying with -t "offset_tune=1, direct_samp=1, digital_agc=1" didn't instantly succeed. However, when capturing the signal and viewing the spectrogram it was clear that the AGC was adding some gain to the signal and the center spike seemed narrower.
I tried again with multiple combinations of offset_tune=1, direct_samp=1, and digital_agc=1, and with different center frequencies. Until, when using a combination of the offset tuning and the digital agc, and the offset center frequency of 433.89 MHz, the first readings came in.
$ rtl_433 -f 433890000 -t "offset_tune=1, digital_agc=1" -S all
It appeared that my "MaxKon" weather station, which shows "FT-0853" on the base station, is deciphered by RTL-433 as a "Cotech-367959" device.
I confirmed this data was coming from the weather station under investigation (instead of a neighbors') by removing the batteries and observing the reports stop until the battery was reinserted.
Each of these readings produced a capture file, and these could also be viewed to show how clearly the software was seeing the signal now.
The Odd Data
The reported data looked like:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time : 2023-11-01 23:46:33
model : Cotech-367959 ID : 3
Battery : 1 Temperature: 164.2 F Humidity : 122 % Rain : 20.4 mm
Wind direction: 181 Wind : 0.0 m/s Gust : 0.0 m/s Integrity : CRC
According to this, the temperature and humidity values were way off the charts.
This was reporting 164.2 F and a humidity of 122% (on a day when the actual temperature was 71 F and 64% humidity.)
Closer Analysis By Hand
Because the suggested decoder "Cotech-367959" had a name that didn't obviously appear on my device, I wanted to be sure that the parsing was correct for my data, and that the high values weren't just an issue with RTL-433.
Viewing the Bits
The RTL-433 command can be modified with a -A flag that instructs it to "Pulse Analyze", which provides useful output for viewing the decoded data.
$ rtl_433 -f 433890000 -t "offset_tune=1, digital_agc=1" -S all -A
Detected OOK package 2023-11-02 01:33:52
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
time : 2023-11-02 01:33:52
model : Cotech-367959 ID : 67
Battery : 1 Temperature: 164.2 F Humidity : 122 % Rain : 0.0 mm Wind direction: 191
Wind : 0.0 m/s Gust : 0.0 m/s Integrity : CRC
Analyzing pulses...
Total count: 110, width: 123.55 ms (30888 S)
Pulse width distribution:
[ 0] count: 93, width: 516 us [496;540] ( 129 S)
[ 1] count: 17, width: 1004 us [1000;1012] ( 251 S)
Gap width distribution:
[ 0] count: 92, width: 456 us [440;484] ( 114 S)
[ 1] count: 17, width: 948 us [944;956] ( 237 S)
Pulse period distribution:
[ 0] count: 84, width: 976 us [972;988] ( 244 S)
[ 1] count: 17, width: 1460 us [1444;1484] ( 365 S)
[ 2] count: 8, width: 1952 us [1948;1964] ( 488 S)
Pulse timing distribution:
[ 0] count: 185, width: 488 us [440;540] ( 122 S)
[ 1] count: 34, width: 976 us [944;1012] ( 244 S)
[ 2] count: 1, width: 10124 us [10124;10124] (2531 S)
Level estimates [high, low]: 15990, 10
RSSI: -0.1 dB SNR: 32.0 dB Noise: -32.1 dB
Frequency offsets [F1, F2]: 4265, 0 (+16.3 kHz, +0.0 kHz)
Guessing modulation: Manchester coding
view at https://triq.org/pdv/#AAB10301E803D0278C808080808080808081919180908180809180808090818080808080808080808080808080808080808091908080808081808080808080808080808080808080918080809080808080808081918090808081919080808080808080808080808190808080808081908081809180809255
Attempting demodulation... short_width: 516, long_width: 0, reset_limit: 960, sync_width: 0
Use a flex decoder with -X 'n=name,m=OOK_MC_ZEROBIT,s=516,l=0,r=960'
[pulse_slicer_manchester_zerobit] Analyzer Device
codes : {127}0029886000017e00010ff4f5fff7f722
*** Saving signal to file g075_433.89M_250k.cu8 (66187 samples, 262144 bytes)
This helps us understand a number of things.
We're told that it appears to be an OOK message, which stands for on-off keying, a simple type of modulation. And it has made the guess that this is using Manchester coding to encode each bit.
We can view the decoded bits on triq.org/pdv/ with the provided link.
And the code is provided at the end:
{127}0029886000017e00010ff4f5fff7f722
That can be converted from hexadecimal to binary:
001010011000100001100000000000000000000101111110000000000000000100001111111101001111010111111111111101111111011100100010
With this we've managed to finally obtain the raw bits being transmitted from the weather station.
Deciphering
The file that handles the decoding for messages of this type lives in rtl_433/src/devices/cotech_36_7959.c as indicated by the model: Cotech-367959 in the output.
The comments in that file helped me map the bits in one of the transmissions from my device to the expected format of the Cotech devices.
1111110101001100001000110000000010000000110101000101000000001010100010000111111110100111101011111111111110111111101110100101
------------AAAABBBBBBBBCDEFGGGGGGGGHHHHHHHHIIIIIIIIJJJJKKKKKKKKKKKKLLLLMMMMMMMMMMMMNNNNNNNNOOOOOOOOOOOOOOOOPPPPPPPPXXXXXXXX
- A : 4 bit: ?? Type code? part of id?, never seems to change
- B : 8 bit: Id, changes when reset
- C : 1 bit: Battery indicator 0 = Ok, 1 = Battery low
- D : 1 bit: MSB of Wind direction
- E : 1 bit: MSB of Wind Gust value
- F : 1 bit: MSB of Wind Avg value
- G : 8 bit: Wind Avg, scaled by 10
- H : 8 bit: Wind Gust, scaled by 10
- I : 8 bit: Wind direction in degrees.
- J : 4 bit: ? Might belong to the rain value
- K : 12 bit: Total rain in mm, scaled by 10
- L : 4 bit: Flag bitmask, always the same sequence: 1000
- M : 12 bit: Temperature in Fahrenheit, offset 400, scaled by 10
- N : 8 bit: Humidity
- O : 16 bit: Sunlight intensity, 0 to 200,000 lumens
- P : 8 bit: UV index (1-15)
- X : 8 bit: CRC, poly 0x31, init 0xc0
The data in my packet matched 1:1 with the format expected here, and the values lined up with what I was seeing in the output in RTL-433 and displayed on the original base station (except for the temperature and humidity.)
This seems to provide increased confidence that the temperature is indeed reporting as 011111111010, and the humidity is reporting as 01111010.
011111111010 is 2042, and according to the formula in the decoder file, (2042 - 400) / 10 equals the temperature we're seeing of 164.2 F. And 01111010 is 122, which is the humidity value we're seeing.
The other values also matched when converted from binary to decimal for the wind average and gust, id, wind direction, and rain values. Everything else appears normal, with the one exception of the temperature and humidity values being too high.
Consulting the Manual
I still had the user manual that came with the weather station. I also found it available online as the Sainlogic FT0835, which I suppose is the actual maker of my weather station, not "MaxKon" which is printed on the side. This makes some sense because the decoder file mentions other "Sainlogic" products.
Consulting the manual to see if it has any information about missing or out-of-range data, it doesn't mention much but at the back it does make the comments:
Allow up to one hour for the sensors to stabilize due to signal filtering. The indoor and outdoor temperature sensors should agree within 2 °C (the sensor accuracy is ± 1 °C).
Use the calibration feature to match the indoor and outdoor temperature to a known source.
Allow up to one hour for the sensors to stabilize due to signal filtering. The indoor and outdoor humidity sensors should agree within 10 % (the sensor accuracy is ± 5 %).
Use the calibration feature to match the indoor and outdoor humidity to a known source.
Unfortunately, it never elaborates anywhere in the manual about what the "calibration feature" is. I tried many combinations of resetting the channel, the batteries, using the reset button, and placing both units indoors together, but nothing seemed to help, and the temperature and humidity data reported in RTL-433 never changed.
Physical Inspection
Because the data was being sent, not just skipped, and not all 1 or all 0, I decided to take a look at the actual temperature/humidity sensor. It is located at the rear of the unit under a sun shade and then inside a further plastic shell.
The sensor was not obviously damaged from the outside. It was only a little dirty and I gave it a dusting and removal of spider webs.
Even Deeper
Because there was no obvious damage on the outside, the case was opened and the board was inspected. This is where some significant dirt and possible issues were located.
This area was cleaned with pressurized air and isopropyl alcohol with a toothbrush.
This cleaning still had no effect on the temperature or humidity values according to RTL-433.
At least digging this deep into the weather station was educational. I was able to see the antenna, the rain gauge, and the magnets and the Hall effect sensors used to detect the rain and wind.
The internal 433 MHz antenna used to transmit the sensor data.
Replacement Sensor
Decided that perhaps the sensor may have gotten damaged from the period of corrosion, or from general exposure over two summers and winters. Decided to replace the component.
I noticed that the sensor seemed to have I2C connections, VCC, GND, SDA, and SCL. With a multimeter I checked the voltage between VCC and GND on the existing sensor and saw it right around 3.0V.
I've since purchased a replacement I2C temperature/humidity sensor rated to work between 2.0 and 5.0 volts, which should fit perfectly here.
What Next
This process has so far been an educational opportunity. It has provided a chance to use the SDR dongle and RTL-433 to find and analyze a signal from misbehaving device.
While trying to understand the missing data, there we were able to dig into the workings of a device decoder for RTL-433 and analyze the bits of a signal by hand. It's also been a chance to crack open a weather station to see how the sensors and components all work.
However, the outdoor temperature and humidity values are still missing...
Now, I'm stuck waiting for the new compatible component to arrive to test the hypotheses that the old one was just damaged from dirt/water ingress.
Stay Tuned:
This page will be updated when the new sensor arrives.