Difference between revisions of "FTL"
(Created page with 'The Nano 2G uses an FTL from Whimory, which has a lot of similarities to the one implemented in openiboot, but seems to be a slightly older version of it. The FTL is divided int...') |
|||
Line 2: | Line 2: | ||
The FTL is divided into two parts, the VFL (virtual flash layer?) and the FTL (flash translation layer). | The FTL is divided into two parts, the VFL (virtual flash layer?) and the FTL (flash translation layer). | ||
+ | |||
+ | == Terminology == | ||
+ | * Logical page (lPage): A logical page (sector) number, as seen by the file system. The FTL block map is used to translate those into vPages. | ||
+ | * Virtual page (vPage): A VFL page number, which is translated to pPages by adding a constant, or by a remap table lookup if that block is marked as bad. | ||
+ | * Physical page (pPage): A physical page number on the flash. | ||
+ | * The same prefixes also apply to blocks. "vBlock" and "lBlock" usually refer to hyperblocks. (Those are on top of the VFL, which handles the bank interleaving.) | ||
+ | * Hyperblock: One block across all banks. | ||
== The VFL == | == The VFL == | ||
Line 34: | Line 41: | ||
uint16_t activecxtblock; | uint16_t activecxtblock; | ||
− | /* Number of the first free page in the active | + | /* Number of the first free page in the active VFL context block */ |
uint16_t nextcxtpage; | uint16_t nextcxtpage; | ||
Line 81: | Line 88: | ||
} __attribute__((packed)); | } __attribute__((packed)); | ||
+ | |||
+ | === VFL mounting procedure === | ||
+ | * Search the last 10% of the flash downwards for a block with at least one of the last 8 pages starting with "DEVICEINFOSIGN\0\0". That page is supposed to also have "BBT\0" at 0x18. | ||
+ | * Look for the BBT in the pages below, according to a scheme specified by that DEVICEINFOSIGN page. In the dumps I've seen, this was always searching the lower (pagesperblock-8) pages in ascending order, until a readable page was found. The data in that page is then used as the lowlevel BBT. | ||
+ | |||
== The FTL == | == The FTL == | ||
− | The FTL is responsible for handling writes that are smaller than the smallest eraseable unit (1 | + | The FTL is responsible for handling writes that are smaller than the smallest eraseable unit (1 "hyperblock") and performs wear leveling. |
=== FTL Context === | === FTL Context === | ||
Line 103: | Line 115: | ||
uint16_t freecount; | uint16_t freecount; | ||
− | /* Index to the first free | + | /* Index to the first free hyperblock in the blockpool ring buffer */ |
uint16_t nextfreeidx; | uint16_t nextfreeidx; | ||
Line 109: | Line 121: | ||
wear. It is incremented on every block erase, and if it | wear. It is incremented on every block erase, and if it | ||
gets too high (300 on writes, 20 on sync), the most and | gets too high (300 on writes, 20 on sync), the most and | ||
− | least worn | + | least worn hyperblock will be swapped (causing an additional |
block write) and the counter will be decreased by 20. */ | block write) and the counter will be decreased by 20. */ | ||
uint16_t swapcounter; | uint16_t swapcounter; | ||
− | /* Ring buffer of currently free | + | /* Ring buffer of currently free hyperblocks. nextfreeidx is the |
index to freecount free ones, the other ones are currently | index to freecount free ones, the other ones are currently | ||
− | allocated for scattered page | + | allocated for scattered page hyperblocks. */ |
uint16_t blockpool[0x14]; | uint16_t blockpool[0x14]; | ||
Line 143: | Line 155: | ||
/* Flag used to indicate that some erase counter pages should be committed | /* Flag used to indicate that some erase counter pages should be committed | ||
− | + | because they were changed more than 100 times since the last commit. */ | |
uint32_t erasedirty; | uint32_t erasedirty; | ||
Line 186: | Line 198: | ||
uint8_t field_8; | uint8_t field_8; | ||
− | /* Type field, 0x40 (data page) or 0x41 (last data page of | + | /* Type field, 0x40 (data page) or 0x41 |
+ | (last data page of hyperblock) */ | ||
uint8_t type; | uint8_t type; | ||
Revision as of 19:41, 2 March 2010
The Nano 2G uses an FTL from Whimory, which has a lot of similarities to the one implemented in openiboot, but seems to be a slightly older version of it.
The FTL is divided into two parts, the VFL (virtual flash layer?) and the FTL (flash translation layer).
Contents
Terminology
- Logical page (lPage): A logical page (sector) number, as seen by the file system. The FTL block map is used to translate those into vPages.
- Virtual page (vPage): A VFL page number, which is translated to pPages by adding a constant, or by a remap table lookup if that block is marked as bad.
- Physical page (pPage): A physical page number on the flash.
- The same prefixes also apply to blocks. "vBlock" and "lBlock" usually refer to hyperblocks. (Those are on top of the VFL, which handles the bank interleaving.)
- Hyperblock: One block across all banks.
The VFL
The VFL is responsible for bad block handling, and emulates a "clean" flash to the FTL. It also contains some information about where to find the FTL context. When a block goes bad, it will be remapped to a spare block near the beginning of the flash. ftl_vfl_cxt_type.remaptable will keep track of those remaps. Each bank has its own independent VFL.
VFL context
/* Keeps the state of the bank's VFL, both on flash and in memory. There is one of these per bank. */ struct ftl_vfl_cxt_type { /* Cross-bank update sequence number, incremented on every VFL context commit on any bank. */ uint32_t usn; /* See ftl_cxt.ftlctrlblocks. This is stored to the VFL contexts in order to be able to find the most recent FTL context copy when mounting the FTL. The VFL context number this will be written to on an FTL context commit is chosen semi-randomly. */ uint16_t ftlctrlblocks[3]; /* Alignment to 32 bits */ uint8_t field_A[2]; /* Decrementing update counter for VFL context commits per bank */ uint32_t updatecount; /* Number of the currently active VFL context block, it's an index into vflcxtblocks. */ uint16_t activecxtblock; /* Number of the first free page in the active VFL context block */ uint16_t nextcxtpage; /* Seems to be unused */ uint8_t field_14[4]; /* Incremented every time a block erase error leads to a remap, but doesn't seem to be read anywhere. */ uint16_t field_18; /* Number of spare blocks used */ uint16_t spareused; /* pBlock number of the first spare block */ uint16_t firstspare; /* Total number of spare blocks */ uint16_t sparecount; /* Block remap table. Contains the vBlock number the n-th spare block is used as a replacement for. 0 = unused, 0xFFFF = bad. */ uint16_t remaptable[0x334]; /* Bad block table. Each bit represents 8 blocks. 1 = OK, 0 = Bad. If the entry is zero, you should look at the remap table to see if the block is remapped, and if yes, where the replacement is. */ uint8_t bbt[0x11A]; /* pBlock numbers used to store the VFL context. This is a ring buffer. On a VFL context write, always 8 pages are written, and it passes if at least 4 of them can be read back. */ uint16_t vflcxtblocks[4]; /* Blocks scheduled for remapping are stored at the end of the remap table. This is the first index used for them. */ uint16_t scheduledstart; /* Probably padding */ uint8_t field_7AC[0x4C]; /* First checksum (addition) */ uint32_t checksum1; /* Second checksum (XOR), there is a bug in whimory regarding this. */ uint32_t checksum2; } __attribute__((packed));
VFL mounting procedure
- Search the last 10% of the flash downwards for a block with at least one of the last 8 pages starting with "DEVICEINFOSIGN\0\0". That page is supposed to also have "BBT\0" at 0x18.
- Look for the BBT in the pages below, according to a scheme specified by that DEVICEINFOSIGN page. In the dumps I've seen, this was always searching the lower (pagesperblock-8) pages in ascending order, until a readable page was found. The data in that page is then used as the lowlevel BBT.
The FTL
The FTL is responsible for handling writes that are smaller than the smallest eraseable unit (1 "hyperblock") and performs wear leveling.
FTL Context
/* Keeps the state of the FTL, both on flash and in memory */ struct ftl_cxt_type { /* Update sequence number of the FTL context, decremented every time a new revision of FTL meta data is written. */ uint32_t usn; /* Update sequence number for user data blocks. Incremented every time a portion of user pages is written, so that a consistency check can determine which copy of a user page is the most recent one. */ uint32_t nextblockusn; /* Count of currently free pages in the block pool */ uint16_t freecount; /* Index to the first free hyperblock in the blockpool ring buffer */ uint16_t nextfreeidx; /* This is a counter that is used to better distribute block wear. It is incremented on every block erase, and if it gets too high (300 on writes, 20 on sync), the most and least worn hyperblock will be swapped (causing an additional block write) and the counter will be decreased by 20. */ uint16_t swapcounter; /* Ring buffer of currently free hyperblocks. nextfreeidx is the index to freecount free ones, the other ones are currently allocated for scattered page hyperblocks. */ uint16_t blockpool[0x14]; /* Alignment to 32 bits */ uint16_t field_36; /* vPages where the block map is stored */ uint32_t ftl_map_pages[8]; /* Probably additional map page number space for bigger chips */ uint8_t field_58[0x28]; /* vPages where the erase counters are stored */ uint32_t ftl_erasectr_pages[8]; /* Seems to be padding */ uint8_t field_A0[0x70]; /* Pointer to ftl_map used by Whimory, not used by us */ uint32_t ftl_map_ptr; /* Pointer to ftl_erasectr used by Whimory, not used by us */ uint32_t ftl_erasectr_ptr; /* Pointer to ftl_log used by Whimory, not used by us */ uint32_t ftl_log_ptr; /* Flag used to indicate that some erase counter pages should be committed because they were changed more than 100 times since the last commit. */ uint32_t erasedirty; /* Seems to be unused */ uint16_t field_120; /* vBlocks used to store the FTL context, map, and erase counter pages. This is also a ring buffer, and the oldest page gets swapped with the least used page from the block pool ring buffer when a new one is allocated. */ uint16_t ftlctrlblocks[3]; /* The last used vPage number from ftlctrlblocks */ uint32_t ftlctrlpage; /* Set on context sync, reset on write, so obviously never zero in the context written to the flash */ uint32_t clean_flag; /* Seems to be unused, but gets loaded from flash by Whimory. */ uint8_t field_130[0x15C]; } __attribute__((packed));
Page metadata (spare bytes)
/* Layout of the spare bytes of each page on the flash */ union ftl_spare_data_type { /* The layout used for actual user data (types 0x40 and 0x41) */ struct ftl_spare_data_user_type { /* The lPage, i.e. Sector, number */ uint32_t lpn; /* The update sequence number of that page, copied from ftl_cxt.nextblockusn on write */ uint32_t usn; /* Seems to be unused */ uint8_t field_8; /* Type field, 0x40 (data page) or 0x41 (last data page of hyperblock) */ uint8_t type; /* ECC mark, usually 0xFF. If an error occurred while reading the page during a copying operation earlier, this will be 0x55. */ uint8_t eccmark; /* Seems to be unused */ uint8_t field_B; /* ECC data for the user data */ uint8_t dataecc[0x28]; /* ECC data for the first 0xC bytes above */ uint8_t spareecc[0xC]; } __attribute__((packed)) user; /* The layout used for meta data (other types) */ struct ftl_spare_data_meta_type { /* ftl_cxt.usn for FTL stuff, ftl_vfl_cxt.updatecount for VFL stuff */ uint32_t usn; /* Index of the thing inside the page, for example number / index of the map or erase counter page */ uint16_t idx; /* Seems to be unused */ uint8_t field_6; /* Seems to be unused */ uint8_t field_7; /* Seems to be unused */ uint8_t field_8; /* Type field: 0x43: FTL context page 0x44: Block map page 0x46: Erase counter page 0x47: "FTL is currently mounted", i.e. unclean shutdown, mark 0x80: VFL context page */ uint8_t type; /* ECC mark, usually 0xFF. If an error occurred while reading the page during a copying operation earlier, this will be 0x55. */ uint8_t eccmark; /* Seems to be unused */ uint8_t field_B; /* ECC data for the user data */ uint8_t dataecc[0x28]; /* ECC data for the first 0xC bytes above */ uint8_t spareecc[0xC]; } __attribute__((packed)) meta; };