Posts for the month of June 2020

Missing EXIF tags could make your android gallery look bad

I recently helped migration to an new android phone. One part of the process was to store old (WhatsApp) stored images on internal storage towards the external storage SD Card.

Installing the new phone revealed an unexpected fall-out of having all the old pictures being sorted as being taken today.

Close inspection revieved Android Media Store uses the EXIF CreateDate to store the moment the picture was taken, missing the tag, it will use the file modification date as backup. How-ever this modification date of course was reset, because the file was copied from internal storage, without preserving the file modification dates.

The first attempt of fixing the issue was by turning off the phone, removing the SD card and fix the modification time of every file, based on the date identifier found in the file e.g. $ touch -a -m -t 201906230000 IMG-20190623-WA001.jpg.

After an re-insert of SD and turning the phone back on, it was not fixing anything. Basically the re-indexing needed to be forced. This is best done by removing all data from the 'Media Storage' System App. Go to Settings -> Apps -> "3 dots button" -> "Show System Apps" -> "Media Storage" -> Storage -> "Clear Data" -> OK. And next reboot the phone and open the images overview on the SD card using Settings -> "Device Care" -> Storage -> "SD Card" -> "Images". Depending on the amount of pictures this could take a while before your device is responsive again.

So finally all pictures and videos are indexed again and the Gallery shows pictures and videos in good order again. All happy right? ... but wait what about Facebook App ... personally I would nuke it and never look back, how-ever some people do not care about being 'the product' and will happily 'pay' for the service by providing lots of personal data. Having this said, I like a good engineering puzzle, so let's get started.

Facebook App (and maybe others) are still showing pictures garbled, which is annoying when for example wanting to upload older pictures, since you have to scroll down a long while before getting to the good stuff again. Apparently facebook does not rely on the metadata generated by the "MediaStore" and generates it's own, how-ever it blindly ignores the file modified date if EXIF data is missing and simply tags the picture as being taken the moment it is indexed. So how-to fix it?

First and foremost we need help, by some very powerful software EXIFTool, this will help us create the required EXIF tags based on the date string found in the filename. So turn off the phone again and plug the SD Card into a computer ones more.

We are going to use this magic:

$ exiftool \  
    "-datetimeoriginal-=" \
    '-datetimeoriginal<${filename;$_=substr($_,4,8)} 00:00' \
    "-createdate-=" \
    '-createdate<${filename;$_=substr($_,4,8)} 00:00' \
    '-filemodifydate<${filename;$_=substr($_,4,8)} 00:00' IMG-*WA*

To explain a little:

  • Do not update DateTimeOriginal EXIF tag if already exists:

"-datetimeoriginal-="

  • Base the DateTimeOriginal EXIF tag on the filename, by selecting the 5th until 12th character in the filename and expanding this with time midnight (00:00). This special rule is required to ensure file named like IMG-20190405-WA-1290.jpg could be parsed without issues, normal syntax is causing 90 to be specified as minutes which causes errors on importing.

'-datetimeoriginal<${filename;$_=substr($_,4,8)} 00:00'

  • Same applies for EXIF CreateDate:
       "-createdate-=" \
        '-createdate<${filename;$_=substr($_,4,8)} 00:00' \
    
  • our previous $ touch equivalent is a nice-to-have yet not required:

'-filemodifydate<${filename;$_=substr($_,4,8)} 00:00' IMG-*WA*

Now, put your SD card back in the phone and turn it back on. Clear your facebook data "Settings -> Apps -> Facebook -> "Clear Data" -> "Ok' and restart your Facebook Apps and wait a long time again. After your phone (literally) cools down it's time to enjoy the result.

Firefox GTK File Upload Dialog not showing previews

I am confused about Firefox under linux not being about to properly preview all images in the "File Upload" dialog. Since camera's do not generate meaning file-name trying to guess/remember/pick the file-name to be used is annoying time consuming.

More-over the behaviour is not consistent, it only seems to happen on large image files, the small ones are generated properly.

Turns out I looking at a work-around for a CVE issue fixed 5 years ago, as seen in Firefox bugzilla issue:1184009.

The work-around consist of disabling the preview for pictures larger than 4096px in height or width). It has been reported multiple times to have it fixed, how-ever all cases being closed, as it was not safe enough.

5 years down the road an new argument comes to table seen at issue:1571984. "They need to re-implement it ourself, since the external library cannot be trusted.", which seems to make sense at first glance.

Leaves me wondering what they mean with "rigging up their own imagelib". Since 10 months has passed since the last comment. I guess it's challenge accepted, I will try to write a patch somehow ....

Removing master references from github repositories

I recently found out the term 'master' as repository name is considered offensive, since it is often used in conjunction with the term 'slave' Which in hindsight is indeed not correct to use.

Since it is not technology limiting how a branch is named, changing is just a matter of making of spending some time on it. If I can make the world a better place by renaming a few of repository trees, it's time well spend.

Since FreeBSD and subversion is using the word trunk together with branches for ages, so I thought it would make a much better alternative.

Switching is a semi-easy, first create and new branch:

$ git checkout -b trunk
Switched to a new branch 'trunk'
$ git push -u origin trunk
Total 0 (delta 0), reused 0 (delta 0)
remote: 
remote: Create a pull request for 'trunk' on GitHub by visiting:
remote:      https://github.com/rickvanderzwet/nagios-plugins/pull/new/trunk
remote: 
To https://github.com/rickvanderzwet/nagios-plugins.git
 * [new branch]        trunk -> trunk
Branch 'trunk' set up to track remote branch 'trunk' from 'origin'.

Deleting old branch:

$ git push origin --delete master
To https://github.com/rickvanderzwet/nagios-plugins.git
 ! [remote rejected]   master (refusing to delete the current branch: refs/heads/master)
error: failed to push some refs to 'https://github.com/rickvanderzwet/nagios-plugins.git'

Whoops make sure to change the GitHub default branch, in mine case found at:

https://github.com/rickvanderzwet/nagios-plugins/branches

And try again:

$ git push origin --delete master
To https://github.com/rickvanderzwet/nagios-plugins.git
 - [deleted]           master

# Delete local branch:
$ git branch -d master
Deleted branch master (was 82e57d4d).

# Update head reference pointer:
$ git remote set-head origin trunk

Dear IBUS don't steal CTRL+ALT+e shortcut in Gnome

I use terminator as my favorite terminal emulator, which uses the shortcut CTRL+ALT+e to open a new window (vertical split).

How-ever recently the key-binding is hijacked by the emojicon hipsters on GNOME to allow inserting emojicons.

To re-claim the key-binding, use ibus-setup and head to the 'Emoji' tab to free the keybinding.

Hacking Samsung SGH-E530 to kids phone

I have a old Samsung SGH-E530 phone which I wanted to convert to play phone. One worry about the phone was the ability to call emergency services (112, 911).

Flight mode was no option, since the phone had to be powered-down, which makes it quite useless as play phone.

Option number two involves drastic measures. Since I will not be needing any GSM capabilities any-more, let's take it out or at least the antenna bit.

Opening the phone was easy peasy, just remove 5 standard philips screws and good to go. Located the antenna, both internal and external and removed a few filter capacitors using tweezers to break them off the board. This will decouple the antenna.

Re-assembly and turning it back in, "Limited service 5 bars". A bit flaky though (holding the phone would make it drop a few bars) how-ever still stable enough to make calls.

Time for a revised plan, time to dive in again. Now locating the Power Amplifier Module (PAM), took the de-soldering station and removed the SKY77328-13 chip from the PCB.

Re-assembly and turning it back in, "Limited service 2 bars"!!! I am located 500 meters from a GSM tower (line of sight), so I am guessing this also helped the process.

Still more to be removed apparently. The "complete RF front end for multi-band GSM and GPRS wireless communications" SI4205-BM is the next canidate for heat-gun removal.

Re-assembly attempt number 3 and finally "No service available", guess I have hit the bulls-eye :-)

Bonus-challenge is setting the date or more precisely the year. The SGH-E530 will not accept the current year (2020), further inspection reveals the date must be set between 1-1-2020 and 31-12-2016. Either no engineer bothered to look this far in the future or a classical case of planned obsolescence.

I would like to set the date correctly to allow learning the day of the week kind of things. Every year the weekday to shift by 1 weekday (365 mod 7), except on a leap year, which is causing the weekday to shift by 2 weekdays.

I order to work around the date restriction I have to improvise. 1-1-2020 was a Wednesday, 6 years earlier, mind the leap-year 2016 in between, 1-1-2014 should be the solution. How-ever 2014 is not a leap-year unlike 2020, thus first of March 2014 is on a Saturday. 1st of March 2020 is located on on a Sunday.

Going back 6 more years to 2008 reveals an other interesting case. The leap-year criteria matches, how-ever located on Tuesday it's incorrect, since there are now 2 leap-years (2012 & 2018) involved.

Going back to 2003 gives me the correct weekday, how-ever no leap year. Going back to 1997, disqualifies as being no leap year. Next in line 1992. Jackpot, first of all a leap-year and secondly 1-1-1992 weekday is Wednesday.

My surprise grows since this behaviour (28 years) is consistent since 1-1-1964 is the next candidate in line. Which is even before UNIX Epoch Time, making it an unlikely candidate to even to be considered :-). I have tried to wrap my head around the math involved to explain it, how-ever I have to leave it "exercise left to the reader".

Back to my pressing matter of setting the right year allowing the weekdays to match has proven to be a rather difficult issue. I can only deal with it by using a work-around. Set the year to 2014 and at 1st of March change the year to 2015. Sounds like next level daylight saving time/summertime :-).