Friday, 24 July 2015

Managing your EEPROMs when upgrading OpenTx

In this post I'll look at EEPROM formats. The knowledge will help you avoid some pitfalls when flashing your transmitter to a newer version of OpenTx.

EEPROM, .EEPE's and data formats

There are three entities which uniquely define the state of your transmitter:

  • The contents of your transmitter's EEPROM
  • Your OpenTx version and build options
  • The contents of the SD card

We're going to concentrate on the first of these, the EEPROM. This is a piece of memory which resides inside your transmitter. It stores all your radio and model settings.

There are several options for creating and managing EEPROM data.

  • You can transfer EEPROM data from your transmitter to a PC 
  • You can create and edit your model setups directly on your transmitter, on in Companion
  • You can transfer EEPROM data from Companion to the PC, overwriting the existing EEPROM data.
  • You can backup/restore EEPROM data from Companion to your hard disk.
  • You can backup/restore EEPROM data from transmitter to SD card.
  • You can backup/restore individual models from EEPROM to SD card.

EEPROM data is saved with an extension of '.bin' or '.eepe'.

EEPROM formats

As more features have been added to OpenTx, so the EEPROM format has evolved. Each format is identified by a version number as follows:
  • OpenTx/Companion9X version 1.x => EEPROM format 215
  • OpenTx/OTX Companion version 2.0.x => EEPROM format 216
  • OpenTx/OTX Companion version 2.1.x => EEPROM format 217
The format of the active EEPROM is shown in the Version menu:

'Version' menu in OpenTx showing OpenTx version 2.0.13, and EEPROM version 216

Some example scenarios:
  • The Taranis X9E comes with OpenTx v2.1 installed, so the EEPROM format is 217
  • A transmitter running OpenTx 2.0.15 will have an EEPROM format 216
  • An eepe file created using Companion9X will contain data in format 215. If the file is opened then saved in OpenTx Companion 2.0, the file will be overwritten in format 216.
As in the last example, OpenTx will convert older EEPROMs as necessary.

Translating between formats

In this section, we'll examine when a format conversion is performed, and some pitfalls to be aware of.

Translation in the transmitter 

After a major point upgrade of OpenTx (e.g. from 2.0 to 2.1), there will be a mismatch between the OpenTx's native format and the format of the existing EEPROM. OpenTx deals with this by translating the EEPROM data to the new format. This is done in situ, during the first boot sequence after the upgrade:

Note that the translation is one-way only - once translated, the EEPROM cannot be downgraded to the earlier format. Because of this, it's imperative to backup your EEPROM and firmware before a major OpenTx upgrade. You can backup directly to the SD card using the RadioSettings.Version menu in your transmitter. Alternatively, use the 'Backup Radio to File' command in Companion and set the file attributes of the saved file to 'ReadOnly' to prevent it from being accidentally overwritten.

Translation in Companion

OpenTx Companion deals with mismatches in a similar way. If you open a .eepe file which was created in an earlier version of OpenTx or Companion, then Companion will translate the data to the newer format. For example, if you use  Companion 2.0.15 to open a .eepe file created in OpenTx 'classic', the data will be translated automatically from version 215 to 216.

Now here's the gotcha: Companion does not issue any warning that the data has been converted. If you do a 'File Save', the .eepe file which was in 215 format will be silently overwritten with data in 216 format. And once data is saved in 216 format, it can't be changed back.

To protect yourself against this: make sure you backup a legacy .EEPE file before opening it in a newer version of Companion - or set the file attributes to 'read-only'.

An EEPROM created in Companion 2.0 (left). When opened in 2.1 (right), format is silently converted from 216 to 217. Only clue is change in byte length.

Your EEPROM data looks corrupted. Why? 

As mentioned above, EEPROM data can never be converted to an older format. However, older versions of Companion do not prevent you from opening newer format files. For example, if you open a format 217 file in Companion 2.0.15 (native format = 216), no warning is issued; the file will be read. However, the data will be corrupted. If you subsequently save the data, it will be corrupted too. [edit: this is fixed in Companion 2.0.18 which will not open files in a newer format]

The upshot is: if the data appears corrupted in Companion, it's probably because you're trying to open an incompatible .EEPE. Make sure that you don't save the corrupted data to a file!

Translations are not always correct!

In order to test the Companion conversion, I took some older .EEPE files and opened them in Companion 2.1.0 (native format 217).
  • In general, Companion 2.1.0 had few difficulties translating 216-format .eepe's. However, translation of data in 215-format was not perfect. Errors included incorrect channel number references in the translated file. 
  • For the data in 215 format, I repeated the translation, this time in two steps: first from 215 to 216 format, the from 216 to 217. I did this by opening the .eepe in Companion 2.0.x then saving; then opening the previous output file in Companion 2.1 and saving again. The resulting .eepe file was much better so this is the method I'd recommend. [TIP: if using multiple versions of Companion, assign a unique icon theme to each ('Settings' menu). That way you can tell which version is open.]
The upshot is: you must check out your model setups carefully before flying.

Migrating your radio settings and models to OpenTx 2.1

Here's the workflow I recommend:
  1. Backup your EEPROM 
  2. Backup your legacy .eepe files 
  3. If you're migrating from OpenTx 2.0, OpenTx 2.1 will automatically translate your EEPROM to the newer format in situ - no further action necessary.
  4. If you're migrating from a version of OpenTx pre-version 2 then I'd recommend converting your EEPROM and .eepe files in two steps- first using Companion 2.0.x and then Companion 2.1.x (as described earlier).
  5. Update sound files to 2.1 (see OpenTx downloads page)
  6. Test your translated setups thoroughly before flying!

Determining the version of a .eepe or .bin

Disclaimer: this info is undocumented, and while the location values work for the majority of files I've tried it on, they aren't 100% reliable:

.eepe files
version is stored in byte location 0x28A (decimal 650).
  • 0x37 = format 215
  • 0x38 = format 216
  • 0x39 = format 217

.bin files.
version is stored in byte location 0x103 (decimal 259)
  • 0xD7 = format 215
  • 0xD8 = format 216
  • 0xD9 = format 217


Mike Sukmawati said...

Once again a great addition to the resources for Taranis. It would be great for you to add to the scope of this blog a bit and incorporate a discussion of the SD card, specifically the need to download different sound files when upgrading from 2.0.x to 2.1.

RC Soar said...

Thanks for pointing this out. Indeed, sound files as stored on SD card (and mirrored on the PC for Companion) must be updated when upgrading from from 2.0 to 2.1 as the indexes have changed. See OpenTx Downloads page