Friday, 24 July 2015

Managing your EEPROMs when upgrading OpenTx

In this post I'll look at EEPROM formats. The knowledge will help you avoid some pitfalls when flashing your transmitter to a newer version of OpenTx.

EEPROM, .EEPE's and data formats

The EEPROM is the area of memory which stores all your radio and model settings. This data, along with OpenTx and the contents of your SD card, uniquely define the behaviour of your particular transmitter.

There are several options for creating and managing EEPROM data. You can edit your model setups directly on your transmitter, or transfer them to a PC and edit them via Companion. You can backup/restore your transmitter EEPROM to the SD card, or to your PC using Companion.

EEPROM data is saved with an extension of '.bin' or '.eepe'.

EEPROM formats

As anyone who has used commercial software like Microsoft Office will know, data formats are always evolving in order to handle the ever increasing sophistication of these programs. It's the same with OpenTx - as more features have been added, so the EEPROM format has evolved. Each format is identified by a version number as follows:
  • OpenTx/Companion9X version 1.x => EEPROM format 215
  • OpenTx/OTX Companion version 2.0.x => EEPROM format 216
  • OpenTx/OTX Companion version 2.1.x => EEPROM format 217
The format of the active EEPROM is shown in the Version menu:

'Version' menu in OpenTx showing OpenTx version 2.0.13, and EEPROM version 216

Some example scenarios:
  • The Taranis X9E comes with OpenTx v2.1 installed, so the EEPROM format is 217
  • A transmitter running OpenTx 2.0.15 will have an EEPROM format 216
  • An eepe file created using Companion9X will contain data in format 215. If the file is opened then saved in OpenTx Companion 2.0, the file will be overwritten in format 216.
As in the last example, OpenTx will convert older EEPROMs as necessary.

Translating between formats

In this section, we'll examine the when a format conversion is required, and some pitfalls to be aware of.

Translation in the transmitter 

After a major point upgrade of OpenTx (e.g. from 2.0 to 2.1), there will be a mismatch between the OpenTx's native format and the format of the existing EEPROM. OpenTx deals with this by translating the EEPROM data to the new format. This is done in situ, during the first boot sequence after the upgrade:




Now here's the gotcha: the translation is one-way only, i.e. once translated, the EEPROM cannot be downgraded to the earlier format. Because of this, it's imperative to backup your EEPROM and firmware before a major OpenTx upgrade. You can backup directly to the SD card using the RadioSettings.Version menu in your transmitter. Alternatively, use the 'Backup Radio to File' command in Companion and set the file attributes of the saved file to 'ReadOnly' to prevent it from being accidentally overwritten.

Translation in Companion

OpenTx Companion deals with mismatches in a similar way. So if you open a .eepe file from an earlier version, Companion will translate the data to the newer working format. For example, if you use  Companion 2.0.15 to open a .eepe file created in OpenTx 'classic', the data will be translated automatically from version 215 to 216.

Now here's another gotcha: Companion does not issue any warning that the data has been converted. If you do a 'File Save', the .eepe file which was in 215 format will be silently overwritten with data in 216 format. And once data is saved in 216 format, it can't be changed back.

The upshot is: backup your legacy .EEPE files before opening them in a newer version of Companion - or set their file attributes to 'read-only'.

An EEPROM created in Companion 2.0 (left). When opened in 2.1 (right), format is silently converted from 216 to 217. Only clue is change in byte length.

Your EEPROM data looks corrupted. Why? 

As mentioned above, EEPROM data cannot be converted to an older format. However, older versions of Companion do not prevent you from opening newer format files. For example, if you open a format 217 file in Companion 2.0.15 (native format = 216), no warning is issued; the file will be read, and the resulting dataset will be corrupted. If you subsequently save the data, it will be corrupted too.

[edit: this is fixed in Companion 2.0.18 which will not open files in a newer format]

So the upshot is: if the data appears corrupted in Companion, it's probably because you're trying to open an incompatible .EEPE. Make sure that you don't save the corrupted data to a file!



Translations are not always correct!

In an ideal world, your model settings will not change as a result of a format conversion in OpenTx or Companion.

In order to test the Companion conversion, I took some older .EEPE files and opened them in Companion 2.1.0 (native format 217).
  • In general, Companion 2.1.0 had few difficulties translating 216-format .eepe's. However, translation of data in 215-format was not perfect. Errors included incorrect channel number references in the translated file. 
  • For the data in 215 format, I repeated the translation, this time in two steps: first from 215 to 216 format, the from 216 to 217. I did this by opening the .eepe in Companion 2.0.x then saving; then opening the previous output file in Companion 2.1 and saving again. The resulting .eepe file was much better so this is the method I'd recommend. [TIP: if using multiple versions of Companion, assign a unique icon theme to each ('Settings' menu). That way you can tell which version is open.]
The upshot is: you must check out your model setups carefully before flying.

Migrating your radio settings and models to OpenTx 2.1

Here's the workflow I recommend:
  1. Backup your EEPROM 
  2. Backup your legacy .eepe files 
  3. If you're migrating from OpenTx 2.0, OpenTx 2.1 will automatically translate your EEPROM to the newer format in situ - no further action necessary.
  4. If you're migrating from a version of OpenTx pre-version 2 then I'd recommend converting your EEPROM and .eepe files in two steps- first using Companion 2.0.x and then Companion 2.1.x (as described earlier).
  5. Update sound files to 2.1 (see OpenTx downloads page)
  6. Test your translated setups thoroughly before flying!

Determining the version of a .eepe or .bin

Disclaimer: this info is undocumented, and while the location values work for the majority of files I've tried it on, they aren't 100% reliable:

.eepe files
version is stored in byte location 0x28A (decimal 650).
  • 0x37 = format 215
  • 0x38 = format 216
  • 0x39 = format 217

.bin files.
version is stored in byte location 0x103 (decimal 259)
  • 0xD7 = format 215
  • 0xD8 = format 216
  • 0xD9 = format 217

2 comments:

Mike Sukmawati said...

Once again a great addition to the resources for Taranis. It would be great for you to add to the scope of this blog a bit and incorporate a discussion of the SD card, specifically the need to download different sound files when upgrading from 2.0.x to 2.1.

RC Soar said...

Thanks for pointing this out. Indeed, sound files as stored on SD card (and mirrored on the PC for Companion) must be updated when upgrading from from 2.0 to 2.1 as the indexes have changed. See OpenTx Downloads page