Sunday, August 27, 2017

Seven years without KDE (or any other desktop environment)

Recently, I just re-read all my old posts from back in the day, and to my astonishment I realized that it's been more than seven years since I threw all the shininess out of my X. So I thought, "heck, why not write another followup?"

Yes, I still use the setup I described back in 2011. Bare-Metal evilwm with bbrun and, as you may have noticed, xbindkeys. That's about it for convenience. As I write this, I have just taken a look at my .xsession. Boy, that takes me back! It still shows all the hash-scars of past experiments. lxpanel. pidgin. yakuake. osd_clock. I used to hack a lot to get my desktop "just so" back then, but I've given (almost) all of it up. There just wasn't any pressure to get in done. What I've still got are the XOSD volume bars, now reduced to one and back in working order (I promised to write about that, didn't I?)

The bottom line is, I still don't miss all the "goodies" the DEs are trying to bait me with. Window management I can do with some simple commands. The two dozen or so windows I usually have open I can keep track of in my mind without a task bar. Start menu? Who needs that? Any application I might need is just a CTRL+ALT+ENTER and a few keystrokes away. I know the commands for what I need. What I don't need, I don't have to have access to. If I once would need it, I would learn the commands. If I want to know the date or time, all I have to do is type "date" or "cal" in xterm. And believe it or not. I still have an old-fashioned clock on the wall above my desk.

All the automount stuff of gnome et al? Never caused me nothing but trouble. Just type "pmount /your/device". How hard can it be? Nowadays the DEs even have auto-execute features for plug-in devices. It's just a matter of time until we get nice auto-exec trojans for linux. Serves people right. There's that German saying that goes "opportunity makes thieves". Same principle applies to such convenience features. Give such a tool to people who don't realize that it CAN be abused, and it WILL be abused because they leave themselves open. None for me, sir, thank you!!!

Long story short, I'll stay in my digital log cabin, thanks. You guys have fun with your shiny toys and blinkenlights. I'm happy in my simple little world.

How to rescue an electronic device that went swimming



They are by no means fool-proof or guaranteed to work, and THEY CAN MAKE MATTERS WORSE, in other words, they can make reparable damage irreparable. Implementing these procedures can void your warranty or insurance.

I guarantee neither correctness nor completeness nor fitness for any purpose whatsoever of these procedures. If you chose to follow them, you do so AT YOUR OWN RISK, and I accept
NO RESPONSIBILITY WHATSOEVER for the results thereof.

If at all possible, you should always avail yourself of a qualified repair service.

Chapter 1: First aid

These procedures apply to any soaked device accident, no matter where you are and if you have a repair service in range or not, with the possible exception of it being just across the street. 

Step 0:

Before retrieving the device from the water, disconnect any mains cables running into it if it is possible to do it safely, or switch of the circuit breaker if it isn't. DO NOT RISK YOUR LIFE. DO NOT TOUCH WET MAINS CABLES WITH YOUR BARE HANDS. DO NOT REACH INTO ANY LIQUID THAT YOU EVEN SUSPECT MIGHT BE CARRYING MAINS VOLTAGE. YOUR DEVICE IS REPLACEABLE. YOU ARE NOT.

If the device bulges or you notice smoke, bubbles or a chemical smell coming from the device,  or if it's hot to the touch, AVOID HANDLING IT: The battery is damaged and may explode, catch fire or vent toxic fumes. Chances are your device is unsalvageable at this point anyways, no don't endanger yourself in vain.

Step 1:

Disconnect any power source and remove any battery. If your battery is non-removable, pray if you got 'em and continue with the following steps immediately and as quickly as possible. Remain mindful of possible battery damage, leakage, overheating and fire. Do not put the device into your pocket, near flammable material or into a closed container.

Step 2: 

Remove any removable parts like back covers, SIM card trays or other attachments. Try to get as much liquid out of it as possible, by rotating and shaking the device repeatedly.

Step 3:

If the device was immersed in salt water or mineral-rich water like a hot spring, rinse it immediately and repeatedly with lots of fresh water. Distilled water is ideal, but almost any kind of reasonably pure water with the possible exception of water from a mineral spring will do. Almost anything is better than salt water corrosion-wise and short-circuit-wise.

Chapter 2: Decision Time

Congrats, you've given a reasonable first aid to your device, the immediate urgency of the problem is now much relieved. Now it's time to take the next


towards saving it.

Option 1: Rush to the hospital
Do you have a qualified repair shop within an hour or two? Take your device there as quickly as possible. None available? Ok, next option.

Option 2: Field expedient surgery 
Can you get your hands on some supplies reasonably quickly? You will need some isopropyl alcohol and distilled water and a baking oven. In a pinch, methylated spirits instead of isopropyl alcohol and any hot, dry place will do. If all else fails, you can leave out step 6 completely and use only distilled water in step 5. Can do? Proceed to step 4. No such luck? Proceed to option 3

Option 3: Supportive care
Can you get none of the above? Not even distilled water? Ok, there's not much more you can do at this point. Proceed to step 5 and use the most pure, mineral-poor water you can find. Skip step 6 and proceed to step 7. After that you have to keep it as dry as possible until you can get it to a repair shop to keep the corrosion in check. If you have one, put the device in a zip lock bag with a sachet of desiccant. If your device is minus it's battery, it still has a decent chance of survival if you get it to a repair shop in a reasonable time before corrosion gets going in earnest. If the battery is non-removable it all depends on how many electrolytes remain in the device and how well you were able to dry it. Take it to a repair shop as soon as you can and hope for the best, but don't get your hopes too high. The leak currents from a charged battery can really play havoc with a device full of water.

Chapter 3: stabilizing the patient

Step 4:

If you are able, and have the tools to do it, disassemble the device. The following procedures CAN be done in an assembled state, but they work much better if done to the individual pieces. Consult the relevant internet self-repair sites for instructions. Keep in mind that you have to put it back together again afterwards, so don't rush into it. Damaging your device or losing parts will make any cleaning success a bitter, pyrrhic victory.

Step 5:

rinse the device repeatedly in a mixture of 70% distilled water and 30% isopropyl alcohol. If no isopropyl alcohol is available, methylated spirit will do, or even good quality Vodka (without added water). The best way to do it is to fill a pan or flat container with the liquid and immerse the device repeatedly. The more liquid flows through the device, the better. The less liquid flows in and out of your device with each rinsing, the longer you should do it. This is to ensure that as little electrolytes as possible remain in the device. Take your time doing this, if the device can't be disassembled, keep at it for at least one hour.

Step 6: 

give the device a few rinses in 90% isopropyl alcohol (or whatever you could lay your hands on) to displace as much of the water as possible. Try to get as much of it out of the device afterwards, because what you get out now you can re-use (isopropyl alcohol isn't cheap) and don't have to evaporate in the next step. Dry right after this step, because isopropyl alcohol is a solvent and extended exposure might damage the glue in the device.

Step 7:

Place your device in a baking oven at 50°C for several hours to dry. Don't accidentally place it in a grill, that would finish it off. If no baking oven is available, any hot, dry place will do in a pinch. If you don't have a hot dry place (because it's winter and you are out hiking), your best case scenario is where you have an unused desiccant bag and a zip lock bag to put it and your device in.  Keep the device on your body to make use of the body heat. If you don't have any desiccant, (and you probably don't), you might try the old "put it in a bag with dry rice" trick - if the rice is VERY dry, at this point this might actually work!

Chapter 4: Recuperation

Even at this point, it is way better to have the device looked over by a qualified repair shop before using it again. If you absolutely have to, or just feel lucky, proceed to step 8. Be aware that if you turn it on where you shouldn't have, it may cause secondary damage, which means it's probably gone for good. But if you absolutely can't wait...

Step 8:

Reassemble, pray, turn on. Either it works, or it doesn't. If it works, do a victory dance and treat yourself to a nice cold one. If it doesn't, you did everything you reasonably could.
To tell the truth, don't expect it to be 100%. Parts like speakers, etc might need replacing afterwards, but you saved most of your device, and, depending on the situation, now that you saved it, it might save you.
To tell even more truth, things aren't cut and dried just yet. Even if the device is looking to be 100% now, it might still conk out on you later from secondary damage that isn't immediately apparent.

So Have. It. Checked. Soonest. Period.

Chapter 5: Some wet device rescue myths, explained.

Placing a wet device in a bag of dry rice can save it.

That will work only if the liquid in question contains little or nothing in the way of electrolytes and non-evaporating additives, like calcium, potassium, sodium, sugar, food coloring, fats, oils, etc, i.e. if the liquid in question was essentially distilled water and/or pure alcohol, in which case the device never was in any real danger in the first place and only wanted drying.

This myth originates from the fact that dry rice is readily available in the household and, if it was stored in a hot, dry climate, has slightly hygroscopic or deliquescent properties, meaning it will absorb moisture from its surroundings, encouraging the evaporation of liquid water. If the device is merely soaked with distilled water, e.g. after step 5, see above, that actually works. There are much better options available than dry rice however, ones that work reliably for starters. Table salt for instance, or if you don't mind a trip to the drug store, sodium hydroxide or lye would be a very good option. If left open to the surrounding air, the sodium hydroxide will actually dissolve in the water it attracts (so don't throw your wet phone in the bag, or you might get a DOOZY of a surprise). Other hygroscopic materials include zinc chloride, calcium chloride, potassium hydroxide and sulphuric acid.

There is an even better option however: Use a purpose-made desiccant, like silica gel for example. it's sold in little sachets that you can seal in a zip lock bag together with your device and let it absorb the moisture. Some of the sachets are even reusable if you bake the moisture out of them in the oven.

And one more point: As detailed in the next paragraph, just drying your device isn't enough. It might give you a false sense of security now and bite you on the ass later.

If you dry your wet device in the oven or with a hair dryer right away, it will be all right 

That might actually seem to work for a while if you live in a perpetually hot and dry climate. The problem here is, the water will evaporate but leave all the substances dissolved in it behind, some of which will certainly be electrolytes. As long as your phone is really, really bone dry, that actually doesn't matter much, because electrolytes need to be dissolved in water to to conduct electricity. However, many of these salts are at least mildly hygroscopic. That means, if the air humidity is only slightly above the "bone dry" mark, the electrolytes WILL start conducting and corroding away your electronics. Either your phone will become a weather forecast device (if it acts wonky, it's going to rain soon) or it will some day soon just fail on you. Period. With hair dryers, baking ovens, microwaves (people actually do that!!!) and so on, you only get the water out. The water isn't the problem however. The stuff dissolved in it is. That's why you are supposed to rinse your phone with distilled water and isopropyl alcohol.

Placing your device in the freezer lowers the conductivity of the water molecules

First and foremost, water molecules don't conduct. Electrolytes do. Second, your freezer isn't all that cold in the general scheme of things. Freezing your device might slow corrosion down a bit, but it sure won't stop it. If that were true, things wouldn't rust in Siberia. What it MIGHT actually do is weaken the internal battery to a point where it doesn't provide quite as much potential to feed the corrosion.

However, freezing causes other problems that are definitely worse than the ones it MIGHT fix. First and foremost, there's water in your device. Water expands when it freezes. If it cracks pavements and even solid rock, do you honestly think it will have a problem ripping those tiny components off your circuit boards? Also LCD screens are notorious for their susceptibility to cold damage. In the olden days, it was recommended you put your phone in your inner pocket when the temperature dips below freezing, because the lcd might take it roughly. Today's devices aren't quite so fragile, but a night in the freezer might very well put paid to the display like the water never could have.

Soaking your device overnight in pure isopropyl alcohol will get all the water out

Yes, it will. It will also dissolve all the glue holding the device together. "Hey, YOU suggested isopropyl alcohol above!!!"

A quick dip is fine, but soaking for extended periods causes more damage than it cures. The main reason for using isopropyl alcohol is that it evaporates more easily than water, and may take away some more grime. The downside is that the very properties that allow it to wash away the grime also allow it to attack the glue in the device. For short periods that isn't a problem if it is dried right after, but an extended soak might make the glue swell up and/or dissolve, leading to problems with stability, bulging or even electrical problems if the glue was used as electrical isolation. In many phones, the screen is glued to the back of the actual front glass. If that glue is dissolved, it often loses it's clarity, leading to foggy ingressions from the rim or worse.

In a pinch, you can rinse your phone in benzene, acetone or turpentine to get the water out

That won't work at all, because to get the water out, you need something that can mix with water, which all the oily substances in this list can't. The surface tension between the different substances together with the surface adhesion and cohesion between the water molecules will actually prevent the oily substances from replacing the water in tiny nooks and crannies, where it is the biggest problem.

If that isn't bad enough for you, many of these substances can attack plastics, rubber and glue, exactly the stuff that holds your device together and your wires apart. If you soak your device in that stuff, you will get surface degradation, bulging, dissolved glue, short circuits, all kinds of fun stuff. And that's not counting the damage the water, which stays in there, will be doing. Not that that bit would matter any more.

Thursday, June 15, 2017

Emulating the Raspberryp Pi with systemd container integration

If you use the Raspberry Pi a lot, you know all about the usual pitfalls, like a new image having ssh disabled, the resolution is really crappy and the image is stuffed full of useless nonsense that nobody who wants to do any real work with the Pi ever needs.

That means that usually you flash your card, you plug in your monitor, keyboard and pointy device, you run your raspi-config, you setup your wifi if you got a Pi 3 or Pi Zero W, you kick out all the useless stuff, you upgrade everything, you install the stuff you actually want to use... aaaand you watch a lot of youtube and do other goofing-off stuff while you wait for the Pi to churn through all the stuff with its tiny processor.
 Sometimes, while you're waiting for a new Pi being delivered, you sit on the flashed SD card for a few days until it gets here so you can get to the waiting in earnest. 

You know, I've got a crazy idea: What if you didn't need the actual Pi to be physically present to configure it? Sure, sure, you can do a lot with manual config edits and stuff. But installing and uninstalling stuff, try doing that with vim! (or emacs, if you're one of that crowd).

The short of it is: It's possible. It turns out there are two ways to do it: a container, or a fully-fledged emulator. It also turns out there are some pitfalls.

How hard can it be?

The former, being fairly simple, is easy to get to run, but has some limitations - if you see the message "unsupported syscall" while you're installing or uninstalling packages, it does not make you happy. Still, since it is the easier one and suitable for most tasks, It is the one I'll show you for now.

Setting the stage

First, you'll want to install a few packages:
# aptitude install qemu qemu-user-static systemd-container binfmt-support
You might also have to manually update things a bit:
# update-binfmts
but usually the install script will have done that for you.

Then, depending on whether you want to use an image or a pre-flashed sd card, you might have to fiddle around with loop devices a bit:
# losetup -f -P --show 2017-01-01-raspbian-jessie.img
That should leave you with several loop devices, including /dev/loop0 for the whole image, as well as /dev/loop0p1 and /dev/loop0p2, guess what those are?

You can then mount the loop device wherever you want it to live:
# mount /dev/loop0p2 -o rw /thisismypi
And you'll also need
# mount /dev/loop0p1 -o rw /thisismypi/boot
especially if you plan to update/upgrade packages that might rely on the stuff in /boot being where it belongs. If you want to work on a pre-flashed SD card, it would look something like this:
# mount /dev/sdd2 -o rw /thisismypi
# mount /dev/sdd1 -o rw /thisismypi/boot
You get the picture.

Transmogrifying the image

The next thing you have to do is cd into there and edit /thisismypi/etc/ and comment out everything in there. That's highly hardware-specific stuff the emulator can't stomach. If you get really weird errors on starting the container, check that you did that first. On that note, if you get weird stuff like no USB support after putting the card into your real Pi, check you UNcommented everything again.

Last but not least, we're planning on using a chroot (that's what a "container" is, when you scrape off the fancy convenience layers and buzzwords), so whatever's not inside /thisismypi won't be visible from the chroot. This is why we have to copy our ARM binary support magic into the container:
# cp /usr/bin/qemu-arm-static /thisismypi/usr/bin
If you are unsure what to copy over, check
update-binfmts --display | grep arm
for the correct name. If you really, really, really don't want anything in your image you don't absolutely need, copy it somewhere else and mount -b the path in.

Let's roll!

There! We're done! Time to jump into our new spaceship and let 'er rip!
# systemd-nspawn -D /thisismypi
Whut? Doesn't look different... Ok, let's try something.
# uname -a
Linux thisismypi 4.4.0-53-generic #74-Ubuntu SMP Fri Dec 2 15:59:10 UTC 2016 armv7l GNU/Linux
Ok, so we're still on Ubuntu, but our architecture is now armv7l. Bingo! Now, let's get crazy:
root@thisismypi:/# su pi
pi@thisismypi:/ $ exit

But does it WORK work?

# apt-get update
Oh! Strange errors! What happen? Somebody set us up the bomb!
Ok, we're not TOTALLY done yet. One thing I've identified so far is that /var/lock is actually a symlink to /run/lock on the Raspberry Pi, and that doesn't get created. So do a
# cd /run/ && mkdir lock
and apt should run fine, at least I didn't have any issues afterwards. Mind, they're most likely there, waiting for me to discover them, so proceed at your own risk!

What does work, what doesn't?

Removing packages works, raspi-config works, but doesn't offer all options. You can of course edit config files at leisure.

Installing packages, hummm... MOSTLY works, but it breaks stuff if it goes too far down onto machine level, which translates to "Your image is now trash". Installing nginx or php should be fine, but I wouldn't try updating kernel modules or other low-level stuff that relies on hardware-specific quirks. Always remember: You're still on your PC hardware, no matter how much it may look like a Pi right now!

Ok, what DOESN'T work? Well for starters, all the custom hardware of the Pi isn't there, so no GPIO, no I2C, no SPI, no other goodies. Then there's everything specific to the broadcom chip, like the bootloader, etc.

To reiterate, it's still running on the very box you're sitting in front of, only qemu gives it the magic powers to run code of a different architecture, and the chroot makes the code think the world ends at the mountpoint. 

What's the worst that could happen?

Now for the million dollar question: Will it run again on the real Pi? Why yes of course it will! The whole exercise would be a bit pointless if it didn't, wouldn't it now? Now, to answer that question bluntly, the worst that could happen is the image not booting. In that case, mount it up again, back up all the irreplaceable stuff you created there, and start over with a fresh image.