Gitiles
Code Review Sign In
review.blissroms.org / platform_build_kati / 5db7f7c46b94424be56db9aa5af4ae537252a703
commit5db7f7c46b94424be56db9aa5af4ae537252a703[log] [tgz]
authorDan Willemsen <dwillemsen@google.com>Wed Feb 22 23:59:06 2017 -0800
committerDan Willemsen <dwillemsen@google.com>Thu Feb 23 18:41:50 2017 -0800
treeb011cb652dfe9a917fa7dacfff8a977f3729f820
parentf8e155865652181a504d7400afd25f35cf158472 [diff]
Randomize cpu affinity

Continue to lock kati to one or two CPUs, but pick one CPU at random,
and pick another CPU next to it. This dramatically speeds up cases where
more than one Kati instance is running at a time.

There's a multiproduct_build tool in Android that attempts to run Kati
on every build configuration present in the tree. My machine has 48
(including hyperthreading) cpu cores, with 64GB of ram. Before this
change, it would take ~12 minutes to build all 59 configurations, 12 at
a time. After this change it takes 3 minutes.

There doesn't seem to be any significant change in timings for
standalone builds. It looks like there was 1-2% difference with my
previous change that chose two completely random CPUs, but choosing two
that are likely hyperthreaded, or at least on the same chip helps.
1 file changed
tree: b011cb652dfe9a917fa7dacfff8a977f3729f820
  1. cmd/
  2. make-c/
  3. testcase/
  4. .gitignore
  5. .travis.yml
  6. affinity.cc
  7. affinity.h
  8. Android.bp
  9. ast.go
  10. AUTHORS
  11. bootstrap.go
  12. buf.go
  13. buf_test.go
  14. command.cc
  15. command.h
  16. CONTRIBUTING.md
  17. CONTRIBUTORS
  18. dep.cc
  19. dep.go
  20. dep.h
  21. depgraph.go
  22. doc.go
  23. eval.cc
  24. eval.go
  25. eval.h
  26. evalcmd.go
  27. exec.cc
  28. exec.go
  29. exec.h
  30. expr.cc
  31. expr.go
  32. expr.h
  33. expr_test.go
  34. file.cc
  35. file.h
  36. file_cache.cc
  37. file_cache.h
  38. fileutil.cc
  39. fileutil.go
  40. fileutil.h
  41. fileutil_bench.cc
  42. find.cc
  43. find.h
  44. find_test.cc
  45. flags.cc
  46. flags.go
  47. flags.h
  48. func.cc
  49. func.go
  50. func.h
  51. func_test.go
  52. INTERNALS.md
  53. io.cc
  54. io.h
  55. LICENSE
  56. loc.h
  57. log.cc
  58. log.go
  59. log.h
  60. m2n
  61. main.cc
  62. make-c.sh
  63. Makefile
  64. Makefile.ckati
  65. Makefile.kati
  66. ninja.cc
  67. ninja.go
  68. ninja.h
  69. ninja_test.cc
  70. ninja_test.go
  71. pack.sh
  72. parser.cc
  73. parser.go
  74. parser.h
  75. pathutil.go
  76. pathutil_test.go
  77. query.go
  78. README.md
  79. regen.cc
  80. regen.h
  81. regen_dump.cc
  82. rule.cc
  83. rule.h
  84. rule_parser.go
  85. rule_parser_test.go
  86. runtest.rb
  87. serialize.go
  88. shellutil.go
  89. shellutil_test.go
  90. stats.cc
  91. stats.go
  92. stats.h
  93. stmt.cc
  94. stmt.h
  95. string_piece.cc
  96. string_piece.h
  97. string_piece_test.cc
  98. stringprintf.cc
  99. stringprintf.h
  100. strutil.cc
  101. strutil.go
  102. strutil.h
  103. strutil_bench.cc
  104. strutil_test.cc
  105. strutil_test.go
  106. symtab.cc
  107. symtab.go
  108. symtab.h
  109. testutil.h
  110. thread_local.h
  111. thread_pool.cc
  112. thread_pool.h
  113. timeutil.cc
  114. timeutil.h
  115. var.cc
  116. var.go
  117. var.h
  118. version.go
  119. version.h
  120. version_unknown.cc
  121. worker.go
README.md

kati

Build Status

kati is an experimental GNU make clone. The main goal of this tool is to speed-up incremental build of Android.

Currently, kati does not offer a faster build by itself. It instead converts your Makefile to a ninja file.

How to use for Android

Now AOSP has kati and ninja, so all you have to do is

% export USE_NINJA=true

All Android's build commands (m, mmm, mmma, etc.) should just work.

How to use for Android (deprecated way)

Set up kati:

% cd ~/src
% git clone https://github.com/google/kati
% cd kati
% make

Build Android:

% cd <android-directory>
% source build/envsetup.sh
% lunch <your-choice>
% ~/src/kati/m2n --kati_stats  # Use --goma if you are a Googler.
% ./ninja.sh

You need ninja in your $PATH.

More usage examples (deprecated way)

"make clean"

% ./ninja.sh -t clean

Note ./ninja.sh passes all parameters to ninja.

Build a specific target

For example, the following is equivalent to "make cts":

% ./ninja.sh cts

Or, if you know the path you want, you can do:

% ./ninja.sh out/host/linux-x86/bin/adb