Setting up emoncms in Docker

Posted on December 15, 2021 by james

As I posted earlier, there’s a github project for setting up emoncms in Docker, but the configuration assumes that a MySQL instance will be set up in a second Docker image and everything built together with docker-compose, which wasn’t what I wanted. I already have MySQL running on the host machine with MythTV and I wanted to use that.

Fortunately, having cloned the project it’s fairly straightforward to get things moving in the right direction. I edited the Dockerfile to add these ENV entries immediately after the FROM, which really isn’t the way to do it and keep your passwords secure, but it will do for me for the moment as I don’t need to share the Docker image anywhere:

ENV MYSQL_HOST=<ip-address>
ENV MYSQL_PORT=3306
ENV MYSQL_DATABASE=emoncms
ENV MYSQL_USER=emoncms
ENV MYSQL_PASSWORD=<password>
ENV REDIS_ENABLED=false
ENV REDIS_HOST=''
ENV REDIS_PORT=6379
ENV REDIS_PREFIX='emoncms'
ENV MQTT_ENABLED=true
ENV MQTT_HOST=<ip-address>
ENV MQTT_USER=emon
ENV MQTT_PASSWORD=<password>
ENV MQTT_BASETOPIC=sensors
ENV PHPFINA_DIR=/var/opt/emoncms/phpfina/
ENV PHPTIMESERIES_DIR=/var/opt/emoncms/phptimeseries/

I changed the topic that emoncms uses to be a bit more intuitive for my use case.

After rebuilding the image I created two local directories, /var/lib/emoncms/phpfina and /var/lib/emoncms/phptimeseries to store some of the data because I think I’d prefer to have it persist over restarts of the container rather than disappear. So my current way to start the package is:

docker run -it -p <ip-address>:80:80 -v /var/lib/emoncms:/var/opt/emoncms emoncms

I’ll put together a systemd configuration for that shortly.

After connecting to the IP address in a browser (which I also need to think about more as perhaps it should have an SSL certificate) I created an account and was then faced with working out what to do next. It really wasn’t clear how I might add sensors of my own into the system. There are templates for some existing devices, but I struggled to find much that was clear regarding how to set up completely new ones. That will be the next step.

Posted in Computing | Tagged , , | Leave a comment

Swimming, 15th December 2021

Posted on December 15, 2021 by james

Having taken Monday off because I didn’t feel great I went back to the pool today not really knowing what to expect. As it turned out, my 50m reps weren’t that bad and I only failed one, though I did get the timing messed up at some point and I can’t recall now if that was rep 10 or rep 11. I’m happy enough at this stage only to have missed one though.

Unfortunately by the time I got to the 25m reps I was absolutely shattered and they were very tough. In fact I didn’t finish the set and missed my target time on three or four. I can live with that though. It’s clear that I was working hard to make the 50m reps and tiredness just crept in towards the end of the session. Hopefully it will be better tomorrow.

Posted in Swimming | Tagged , | Leave a comment

The power monitoring plan is coming together

Posted on December 14, 2021 by james

As I may want to use power consumption data (and other information, such as data from the weather station) in a number of different applications, I’ve come to the conclusion that MQTT may be a way forward, allowing those applications that way the data to subscribe to the relevant topics to get it. Specifically, I’m looking at using Mosquitto. I already have a server running all the time for MythTV, so that seems like a logical place to run it.

I’m a little behind the curve with containers and related technology so I thought I’d explore the possibility of using Docker for some of the components I need to get everything up and running. I’m thinking I’ll do that for anything that doesn’t need direct access to hardware.

Happily there’s an existing Docker image for Mosquitto that I can use which makes things fairly straightforward, though I did have one little niggle getting it to run initially. The documentation on the linked page suggests running it using:

docker run -it -p 1883:1883 -p 9001:9001 -v mosquitto.conf:/mosquitto/config/mosquitto.conf -v /mosquitto/data -v /mosquitto/log eclipse-mosquitto

but when I did that I was getting errors about mosquitto.conf not being a directory, so I checked to see if there were any other files in the image’s /mosquitto/config directory (there weren’t) and decided to change the command to:

docker run -it -p 1883:1883 -p 9001:9001 -v /etc/mosquitto:/mosquitto/config -v /mosquitto/data -v /mosquitto/log eclipse-mosquitto

That allowed me to have a copy of the config file on the host system that I could work with, but also allowed me to add acl files and password files in the same place.

Then I realised that to test from the host system (currently Ubuntu 20.04) I also needed to install the mosquitto package which wants to put its config files in the same place, so I moved mine to /etc/mosquitto.docker and updated the docker command accordingly:

/usr/bin/docker run --rm -t -p 1883:1883 -p 9001:9001 -v /etc/mosquitto.docker:/mosquitto/config -v /mosquitto/data -v /mosquitto/log eclipse-mosquitto

then installed the mosquitto and mosquitto-clients packages.

Testing looked good, so the final stage was to get the container up and running at boot time using systemd. As I may wish to move the container elsewhere at some point in the future I bound a spare IP address to the host server’s network interface and came up with the following configuration which I put in /etc/systemd/system/docker-mosquitto.service

[Unit]
Description=Mosquitto Container
After=docker.service
Requires=docker.service

[Service]
TimeoutStartSec=0
Restart=always
ExecStartPre=-/usr/bin/docker stop eclipse-mosquitto
ExecStartPre=-/usr/bin/docker rm eclipse-mosquitto
ExecStartPre=/usr/bin/docker pull eclipse-mosquitto
ExecStart=/usr/bin/docker run --rm -t -p <ip-address>:1883:1883 -p <ip-address>:9001:9001 -v /etc/mosquitto.docker:/mosquitto/config -v /mosquitto/data -v /mosquitto/log eclipse-mosquitto

[Install]
WantedBy=multi-user.target

In truth this probably isn’t perfect as it may not restart properly should the instance fail, but right now I’m more concerned about having it start cleanly at boot time than what happens if something breaks.

sudo systemctl daemon-reload
sudo systemctl start docker-mosquitto

started everything and testing again pointed to everything being good, so I think for the time being I’m done.

I’m looking at using emoncms from the Open Energy Monitor project to display power data as it appears to be able to grab inputs from MQTT, so getting that up and running together and probably having to write some code (the horror!) to read and submit the data are probably the next steps. It looks as though there’s a project to create a Docker image for that too, though it seems to be tied up with using MySQL in a second image which I don’t want to do as I already have MySQL running on the host server for MythTV and I’d prefer to use that for my database.

Posted in Computing | Tagged , | Leave a comment

Tonight Matthew, I am Dremel repair man

Posted on December 13, 2021 by james

I posted a few days ago that my Dremel 4000 had died and that I suspected the thermal fuse to be the cause. Here’s how I fixed it.

The first steps to getting inside the case are to remove the folding clip, the collet and the collar on the end of the body behind the collet. It may be possible to remove the collar without removing the collet, but when it’s so easy to get the collet off, why make things difficult?

The five Torx screws holding the case together can then be removed and that side of the case gently prised off. There are a few small parts that may go “ping” at this point. In particular the two brush holder covers (small black plastic caps about halfway down the body of the tool) and the brushes they retain, and the locking button for the shaft near the nose of the tool. So a wise person probably does this bit where such parts won’t disappear down the back of a cupboard and never be seen again 🙂

Hopefully that leaves you with a view somewhat similar to this:

At this point it’s probably easiest to remove the lower brush cover and the brush itself (the ones on the opposite side to the speed control and power switch).

This allows comfortable access to the brush holder and the attached fuse (the silver coloured part with the yellow end in this photo).

A multimeter placed across the brush holder and the wire coming out of the yellow end of the fuse in continuity-testing mode should indicate whether the fuse has failed or not. If it conducts then the problem lies elsewhere. In my case it didn’t. The fuse and brush holder are supplied as a single part when ordering spares which makes replacement quite straightforward. The fuse connects to the motor with a small spade connector that can be pulled out carefully and then the two parts just lift away.

The spares sites I found for ordering replacements had exploded diagrams of the tool with all the parts numbered. It wasn’t entirely obvious from the description alone that one brush holder just has a fly lead with a spade connector on the end whilst the other has the thermal fuse. They were both just described as “brush holder”. It’s therefore important to use the diagram to make sure you get the right part. Another clue is that the one with the thermal fuse is the more expensive of the two. I’d hazard a guess that the other brush holder might actually work were it to be fitted instead, but you’d end up with a Dremel with no protection from overheating. If you’re the kind of person who treats their tools with care and good sense then that might not be a problem. If you’re the kind of cack-handed ape whose screwdrivers often double as removal tools for oil filters then you’re probably better off keeping the thermal fuse 😀

Reassembly with the new part is, as they say in all the good servicing manuals, just the reverse of disassembly 🙂 Take care not to trap wires between the brush holder and body of the tool though. They should fit into the gaps in the moulding. And be sure to get the orientation of the brush correct when it is put back in, so it makes proper contact with the commutator. The end of the brush is curved where it fits against the surface of the commutator.

Posted in Workshop | Tagged | 1 Comment

Piano, four days in

Posted on December 12, 2021 by james

I’ve been practising every day and the simple stuff seems to be coming more easily now, but it’s far too early to make a decision between the two books I’ve been reading to learn. Andrew Furmanczyk’s videos I’m losing faith with a little. Partly I think he’s just not my style, but also I’m not yet sure where it’s all going — there seems to be a lot of information, but not a lot of “glue” to put it all together that makes for meaningful learning. I’ll stick with them for a while yet though.

One other source of online lessons I’ve found is another set of YouTube videos by Bill Hilton. He has a “How to play piano for beginners” playlist as well as a lot of other stuff that might come in handy one day. I’ve only watched the first one so far, but he seems more direct and perhaps a little more to my taste. I’ll certainly watch a few more to see how they go.

Posted in Learning piano, Projects | Leave a comment

Swimming, 12th December 2021

Posted on December 12, 2021 by james

Another no-so-great performance today. Better than Thursday, but not at the level I was at prior to that. Tiredness still affecting me I think, or my daughter’s cold. I stuck with it though, just because I didn’t want to leave the session early.

Posted in Swimming | Tagged , | Leave a comment

Swimming, 9th December 2021

Posted on December 9, 2021 by james

Well, today was just awful 🙁

I’m still not sleeping well and I think today was the day the tiredness chose to make itself known. Either that or I’m getting the cold that my daughter currently has and my body just hasn’t let me know yet.

I barely had a single 50m rep below my target time and had failed three by the time I’d been in the water half an hour so decided that wimping out was the better part of valour and went home.

Perhaps the cause will become obvious over the next few days.

Posted in Swimming | Tagged , | Leave a comment

Piano is not my forte. Yet.

Posted on December 8, 2021 by james

I’ve dabbled with musical instruments over the years, but I’ve never really gone anywhere near a piano and I’ve decided I quite like the idea of learning. Happily this weekend I discovered that we still have one of the keyboards my son used fairly early on. It’s a Yamaha PSR-E353, so it only has five octaves rather than the full piano complement, but there’s always my son’s electric piano if/when I reach the stage of needing more notes. I just won’t have access to that out of term time, so the Yamaha is more convenient for the time being.

As for how to learn, I’m dithering at the moment. I’ve been watching some of the videos on Andrew Furmanczyk’s YouTube channel. I find the music theory stuff good because I don’t know a lot about that, but the first “learning piano” one is perhaps a bit childish in places and rambles a little. I’ll stick with it though and see how the later lessons are. From the point of view of being an author of open-source software, I certainly appreciate that they exist and are available for free.

I’ve also bought a couple of books:

I picked the spiral-bound versions so they’ll sit flat on a music stand or desk. Going through a few reviews they seemed to be the closest match to the kind of thing I’d be after, so I’ll see how I get on with them as well and post a review when I have a good feel for what’s what. Perhaps I’ll review Andrew Furmanczyk’s lessons on a video-by-video basis too.

Oh! I nearly forgot… The other important purchase I made was a set of headphones. I don’t need to inflict the sound of me practising on the rest of my family just yet 😀

Posted in Learning piano | Leave a comment

Swimming, 8th December 2021

Posted on December 8, 2021 by james

I’ve not slept well for the last two nights, so once again I wasn’t expecting much when I went to the pool today and the situation wasn’t improved when four other swimmers joined the fast lane, all about ten seconds a length slower than my usual speed.

In fact, things actually went pretty well for the 50m reps and I had no failures, though I did have a couple of reps slower than my 43s target time when I was obstructed at the turn by one of the other swimmers. Whilst I’m happy with the performance therefore, I’m not going to count it as a “completed” set.

My 25m reps were a real struggle though. By that point I felt very tired and ended up having to skip a couple due to failures. Not great, but I understand the reason so I’m not going to get stressed about it.

Now I’ll just have to see how things go tomorrow. Hopefully I can get some decent sleep tonight.

Posted in Swimming | Tagged , | Leave a comment

The rain gauge is playing up again :(

Posted on December 6, 2021 by james

When I put it back up things seemed to be going well, but over the last few days the rain gauge has been giving some very strange results. I suspect it has been counting bucket tips when there aren’t any, and over-counting those there are.

Initially I think I’m going to pop to top off and remove some of the magnets I added to try to get it to trigger the reed switch more positively. It is perhaps possible that the switch is getting triggered more than once per tip, but perhaps the buckets might also be getting caught up somewhere that is causing the switch to trigger when it shouldn’t. In the last hour for example, almost seventy tips have been counted when it hasn’t been raining (and nor is it windy enough that the gauge might be rocking and counting extra tips).

Another oddity is that despite starting with both counters in the DS2423 at zero, counter A is now sixty greater than counter B. I’m really not sure I understand how that happens when the two inputs are tied together.

Periods of heavy rain are forecast for tomorrow with “Storm Barra” following closely on the heels of “Storm Arwen”, but hopefully I can find a few dry minutes to open the unit up and see what’s going on.

Posted in Environment, Projects | Tagged | Leave a comment