releasetools: Reduce memory footprint for BBOTA generation.
The major issue with the existing implementation is unnecessarily
holding too much data in memory, such as HashBlocks() which first reads
in *all* the data to a list before hashing. We can leverage generator
functions to stream such operations.
This CL makes the following changes to reduce the peak memory use.
- Adding RangeSha1() and WriteRangeDataToFd() to Image classes. These
functions perform the operations on-the-fly.
- Caching the computed SHA-1 values for a Transfer instance.
As a result, this CL reduces the peak memory use by ~80% (e.g. reducing
from 5.85GB to 1.16GB for the same incremental, as shown by "Maximum
resident set size" from `/usr/bin/time -v`). It also effectively
improves the (package generation) performance by ~30%.
Bug: 35768998
Bug: 32312123
Test: Generating the same incremental w/ and w/o the CL give identical
output packages.
Change-Id: Ia5c6314b41da73dd6fe1dbe2ca81bbd89b517cec
2 files changed