Merge "Fix perf tests for cold startup." into nyc-dev