how to distribute android 10/11 ?

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

how to distribute android 10/11 ?

alexander zhu
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.


I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

Orion Hodson
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <[hidden email]> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.


I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

Dan Willemsen-2
The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

RBE is the replacement, but it's still evolving at the moment. Many of our CI builds have switched to it (they never used Goma), but only a handful of developers have used it so far. If you're interested in this, I'd highly recommend asking your TAM about it (or if you don't have one, contact me and I can try to route your request to the right team).

For what we're doing (other than Goma/RBE), the majority of our CI builds are done on n1-standard-64 machines with SSD persistent disks. Depending on your limitations/requirements, the newer C2/N2 machine families may be more efficient. For local developers, we're generally buying fast desktops -- 2x Intel Xeon GOLD 6154 processor (18-core Skylake, up to 3.70 GHz) with 192 GB RAM and a few TB of SSDs.

- Dan

On Sun, Mar 1, 2020 at 11:52 PM 'Orion Hodson' via Android Building <[hidden email]> wrote:
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <[hidden email]> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.


I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CALQgHdk6-TjDLvmb29DuZc-SB5L76BdJvxU-wUJKVRNkFVq%3DwA%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

Dan Willemsen-2
Oh, and for developer builds (and many frequent CI builds), we're almost always using incremental builds, so the clean build times are somewhat less limiting. If you're often doing clean builds or very large builds, that's probably one of the first things to look at.

- Dan

On Tue, Mar 3, 2020 at 1:57 PM Dan Willemsen <[hidden email]> wrote:
The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

RBE is the replacement, but it's still evolving at the moment. Many of our CI builds have switched to it (they never used Goma), but only a handful of developers have used it so far. If you're interested in this, I'd highly recommend asking your TAM about it (or if you don't have one, contact me and I can try to route your request to the right team).

For what we're doing (other than Goma/RBE), the majority of our CI builds are done on n1-standard-64 machines with SSD persistent disks. Depending on your limitations/requirements, the newer C2/N2 machine families may be more efficient. For local developers, we're generally buying fast desktops -- 2x Intel Xeon GOLD 6154 processor (18-core Skylake, up to 3.70 GHz) with 192 GB RAM and a few TB of SSDs.

- Dan

On Sun, Mar 1, 2020 at 11:52 PM 'Orion Hodson' via Android Building <[hidden email]> wrote:
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <[hidden email]> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.


I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CALQgHdmsfJ1HgwZKF1%2Bw1wXUsEOP9%2B6EKTV-roihEOXycWj_kw%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

alexander zhu
hi,Dan

I'm so glad to get your message.specially, the "buying fast desktops "parts . I did the same thing . please give my request to the right team. So many thanks to you . I will continue to follow the RBE things. 

在 2020年3月4日星期三 UTC+8上午6:00:40,Dan Willemsen写道:
Oh, and for developer builds (and many frequent CI builds), we're almost always using incremental builds, so the clean build times are somewhat less limiting. If you're often doing clean builds or very large builds, that's probably one of the first things to look at.

- Dan

On Tue, Mar 3, 2020 at 1:57 PM Dan Willemsen <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">dwill...@...> wrote:
The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

RBE is the replacement, but it's still evolving at the moment. Many of our CI builds have switched to it (they never used Goma), but only a handful of developers have used it so far. If you're interested in this, I'd highly recommend asking your TAM about it (or if you don't have one, contact me and I can try to route your request to the right team).

For what we're doing (other than Goma/RBE), the majority of our CI builds are done on <a href="https://cloud.google.com/compute/docs/machine-types#n1_machine_type" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://cloud.google.com/compute/docs/machine-types#n1_machine_type&#39;;return true;" onclick="this.href=&#39;https://cloud.google.com/compute/docs/machine-types#n1_machine_type&#39;;return true;">n1-standard-64 machines with SSD persistent disks. Depending on your limitations/requirements, the newer C2/N2 machine families may be more efficient. For local developers, we're generally buying fast desktops -- 2x Intel Xeon GOLD 6154 processor (18-core Skylake, up to 3.70 GHz) with 192 GB RAM and a few TB of SSDs.

- Dan

On Sun, Mar 1, 2020 at 11:52 PM 'Orion Hodson' via Android Building <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com> wrote:
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

<a href="https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8&#39;;return true;">https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">mooo...@...> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.

I noticed there had topic long time ago : <a href="https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J&#39;;return true;">https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J

I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.<a rel="nofollow" href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884" style="color:rgb(3,102,214);font-family:-apple-system,BlinkMacSystemFont,&quot;Segoe UI&quot;,Helvetica,Arial,sans-serif,&quot;Apple Color Emoji&quot;,&quot;Segoe UI Emoji&quot;;font-size:14px" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fbugs.debian.org%2Fcgi-bin%2Fbugreport.cgi%3Fbug%3D926884\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGEaBbu5ktMciywP-z8_DDZ0_zy4A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fbugs.debian.org%2Fcgi-bin%2Fbugreport.cgi%3Fbug%3D926884\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGEaBbu5ktMciywP-z8_DDZ0_zy4A&#39;;return true;">https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com
To unsubscribe from this group, send email to
<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com
For more options, visit this group at
<a href="http://groups.google.com/group/android-building?hl=en" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;" onclick="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;">http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com
To unsubscribe from this group, send email to
<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com
For more options, visit this group at
<a href="http://groups.google.com/group/android-building?hl=en" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;" onclick="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;">http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/81098239-e8dd-4bca-9061-1882eca234f3%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

alexander zhu
In reply to this post by Dan Willemsen-2
hi,
Dan,

does RBE is your GCP service?  i want to know how it works

在 2020年3月4日星期三 UTC+8上午6:00:40,Dan Willemsen写道:
Oh, and for developer builds (and many frequent CI builds), we're almost always using incremental builds, so the clean build times are somewhat less limiting. If you're often doing clean builds or very large builds, that's probably one of the first things to look at.

- Dan

On Tue, Mar 3, 2020 at 1:57 PM Dan Willemsen <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">dwill...@...> wrote:
The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

RBE is the replacement, but it's still evolving at the moment. Many of our CI builds have switched to it (they never used Goma), but only a handful of developers have used it so far. If you're interested in this, I'd highly recommend asking your TAM about it (or if you don't have one, contact me and I can try to route your request to the right team).

For what we're doing (other than Goma/RBE), the majority of our CI builds are done on <a href="https://cloud.google.com/compute/docs/machine-types#n1_machine_type" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://cloud.google.com/compute/docs/machine-types#n1_machine_type&#39;;return true;" onclick="this.href=&#39;https://cloud.google.com/compute/docs/machine-types#n1_machine_type&#39;;return true;">n1-standard-64 machines with SSD persistent disks. Depending on your limitations/requirements, the newer C2/N2 machine families may be more efficient. For local developers, we're generally buying fast desktops -- 2x Intel Xeon GOLD 6154 processor (18-core Skylake, up to 3.70 GHz) with 192 GB RAM and a few TB of SSDs.

- Dan

On Sun, Mar 1, 2020 at 11:52 PM 'Orion Hodson' via Android Building <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com> wrote:
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

<a href="https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8&#39;;return true;">https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">mooo...@...> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.

I noticed there had topic long time ago : <a href="https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J&#39;;return true;">https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J

I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.<a rel="nofollow" href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884" style="color:rgb(3,102,214);font-family:-apple-system,BlinkMacSystemFont,&quot;Segoe UI&quot;,Helvetica,Arial,sans-serif,&quot;Apple Color Emoji&quot;,&quot;Segoe UI Emoji&quot;;font-size:14px" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fbugs.debian.org%2Fcgi-bin%2Fbugreport.cgi%3Fbug%3D926884\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGEaBbu5ktMciywP-z8_DDZ0_zy4A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fbugs.debian.org%2Fcgi-bin%2Fbugreport.cgi%3Fbug%3D926884\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGEaBbu5ktMciywP-z8_DDZ0_zy4A&#39;;return true;">https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com
To unsubscribe from this group, send email to
<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com
For more options, visit this group at
<a href="http://groups.google.com/group/android-building?hl=en" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;" onclick="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;">http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com
To unsubscribe from this group, send email to
<a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com
For more options, visit this group at
<a href="http://groups.google.com/group/android-building?hl=en" target="_blank" rel="nofollow" onmousedown="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;" onclick="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;">http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="kCS1EacfAwAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">android-...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/b0e9cc79-330b-441a-92de-89a78cf861d4%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

Duslabo
In reply to this post by Dan Willemsen-2
Cool,

Does anybody did benchmarking of aosp clean build on AMD Reyzen threadripper?

Regards,
Duslabo

On Wed, 4 Mar 2020 at 03:27, 'Dan Willemsen' via Android Building <[hidden email]> wrote:
The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

RBE is the replacement, but it's still evolving at the moment. Many of our CI builds have switched to it (they never used Goma), but only a handful of developers have used it so far. If you're interested in this, I'd highly recommend asking your TAM about it (or if you don't have one, contact me and I can try to route your request to the right team).

For what we're doing (other than Goma/RBE), the majority of our CI builds are done on n1-standard-64 machines with SSD persistent disks. Depending on your limitations/requirements, the newer C2/N2 machine families may be more efficient. For local developers, we're generally buying fast desktops -- 2x Intel Xeon GOLD 6154 processor (18-core Skylake, up to 3.70 GHz) with 192 GB RAM and a few TB of SSDs.

- Dan

On Sun, Mar 1, 2020 at 11:52 PM 'Orion Hodson' via Android Building <[hidden email]> wrote:
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <[hidden email]> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.


I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CALQgHdk6-TjDLvmb29DuZc-SB5L76BdJvxU-wUJKVRNkFVq%3DwA%40mail.gmail.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CAEJYLT6G-3Sh2zO6dkScKM_TRXG%3Dwm0shbxNnanVOBbL%3DwgBqg%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

alexander zhu
In reply to this post by alexander zhu
hi,

Dan,

I got one ppt, which is 'Engineering productivity talk' in 2020,which pages mention you. I read it and go through the pages and overview(https://cloud.google.com/sdk/gcloud/reference/alpha/remote-build-execution). It looks like the result is what I'm looking for.

result.png




I have some quetions:

1. does the result use Android R master? what is the number of “-j” you use? because I faced one problem, which is the larger the "-j" is the java OOM error comes out.In my case , 32U128G server,I only can set -j72. if the number bigger than this number there is big chance to face the JAVA OOM(system is out of resource).
2. Does RBE build the binary(.o or .so or jar) to put it into remote cache, and next time some one build ,the new one will use the binary instead of real compiling ? how does this work? each action has one md5 ?
3. how to solve java action in build ? I tried distcc ,in some modules it can work, but java part. 
4. so many details I want to know .I want to build one thing like this to solve my Android build so slow problem

在 2020年3月20日星期五 UTC+8上午12:07:39,alexander zhu写道:
hi,
Dan,

does RBE is your GCP service?  i want to know how it works

在 2020年3月4日星期三 UTC+8上午6:00:40,Dan Willemsen写道:
Oh, and for developer builds (and many frequent CI builds), we're almost always using incremental builds, so the clean build times are somewhat less limiting. If you're often doing clean builds or very large builds, that's probably one of the first things to look at.

- Dan

On Tue, Mar 3, 2020 at 1:57 PM Dan Willemsen <[hidden email]> wrote:
The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

RBE is the replacement, but it's still evolving at the moment. Many of our CI builds have switched to it (they never used Goma), but only a handful of developers have used it so far. If you're interested in this, I'd highly recommend asking your TAM about it (or if you don't have one, contact me and I can try to route your request to the right team).

For what we're doing (other than Goma/RBE), the majority of our CI builds are done on <a href="https://cloud.google.com/compute/docs/machine-types#n1_machine_type" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://cloud.google.com/compute/docs/machine-types#n1_machine_type&#39;;return true;" onclick="this.href=&#39;https://cloud.google.com/compute/docs/machine-types#n1_machine_type&#39;;return true;">n1-standard-64 machines with SSD persistent disks. Depending on your limitations/requirements, the newer C2/N2 machine families may be more efficient. For local developers, we're generally buying fast desktops -- 2x Intel Xeon GOLD 6154 processor (18-core Skylake, up to 3.70 GHz) with 192 GB RAM and a few TB of SSDs.

- Dan

On Sun, Mar 1, 2020 at 11:52 PM 'Orion Hodson' via Android Building <[hidden email]> wrote:
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

<a href="https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8&#39;;return true;">https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/eOtBzosZtd8

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <[hidden email]> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.

I noticed there had topic long time ago : <a href="https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J&#39;;return true;">https://groups.google.com/forum/#!searchin/android-building/distcc%7Csort:date/android-building/_YJNwvq6FtY/EENyTx4Elk4J

I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.<a rel="nofollow" href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884" style="color:rgb(3,102,214);font-family:-apple-system,BlinkMacSystemFont,&quot;Segoe UI&quot;,Helvetica,Arial,sans-serif,&quot;Apple Color Emoji&quot;,&quot;Segoe UI Emoji&quot;;font-size:14px" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fbugs.debian.org%2Fcgi-bin%2Fbugreport.cgi%3Fbug%3D926884\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGEaBbu5ktMciywP-z8_DDZ0_zy4A&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fbugs.debian.org%2Fcgi-bin%2Fbugreport.cgi%3Fbug%3D926884\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGEaBbu5ktMciywP-z8_DDZ0_zy4A&#39;;return true;">https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
<a href="http://groups.google.com/group/android-building?hl=en" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;" onclick="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;">http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
<a href="http://groups.google.com/group/android-building?hl=en" rel="nofollow" target="_blank" onmousedown="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;" onclick="this.href=&#39;http://groups.google.com/group/android-building?hl\x3den&#39;;return true;">http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com?utm_medium=email&amp;utm_source=footer" rel="nofollow" target="_blank" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/android-building/CAAttue0u%2BuqgwNO4ggmnWsYS%2B8FmL8xURN-Xqt9sw9d1aLENQQ%40mail.gmail.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/b749f4fb-6998-4f46-bc98-3bd7a681540f%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

唐木村
In reply to this post by Dan Willemsen-2
@Dan Willemsen

>>> The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

whether goma not work well for android build ??? and now 2021 year support better ???
https://chromium.googlesource.com/infra/goma/client/
https://chromium.googlesource.com/infra/goma/server/

在2020年3月4日星期三 UTC+8 上午5:57:32<Dan Willemsen> 写道:
The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

RBE is the replacement, but it's still evolving at the moment. Many of our CI builds have switched to it (they never used Goma), but only a handful of developers have used it so far. If you're interested in this, I'd highly recommend asking your TAM about it (or if you don't have one, contact me and I can try to route your request to the right team).

For what we're doing (other than Goma/RBE), the majority of our CI builds are done on n1-standard-64 machines with SSD persistent disks. Depending on your limitations/requirements, the newer C2/N2 machine families may be more efficient. For local developers, we're generally buying fast desktops -- 2x Intel Xeon GOLD 6154 processor (18-core Skylake, up to 3.70 GHz) with 192 GB RAM and a few TB of SSDs.

- Dan

On Sun, Mar 1, 2020 at 11:52 PM 'Orion Hodson' via Android Building <[hidden email]> wrote:
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <[hidden email]> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.


I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/6062ea95-ea49-483d-8e32-675bbf2a954en%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

Dan Willemsen-2
GOMA support has been removed on the AOSP master branch, and has been replaced with RBE.

- Dan

On Wed, Feb 3, 2021 at 9:30 AM 唐木村 <[hidden email]> wrote:
@Dan Willemsen

>>> The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

whether goma not work well for android build ??? and now 2021 year support better ???

在2020年3月4日星期三 UTC+8 上午5:57:32<Dan Willemsen> 写道:
The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

RBE is the replacement, but it's still evolving at the moment. Many of our CI builds have switched to it (they never used Goma), but only a handful of developers have used it so far. If you're interested in this, I'd highly recommend asking your TAM about it (or if you don't have one, contact me and I can try to route your request to the right team).

For what we're doing (other than Goma/RBE), the majority of our CI builds are done on n1-standard-64 machines with SSD persistent disks. Depending on your limitations/requirements, the newer C2/N2 machine families may be more efficient. For local developers, we're generally buying fast desktops -- 2x Intel Xeon GOLD 6154 processor (18-core Skylake, up to 3.70 GHz) with 192 GB RAM and a few TB of SSDs.

- Dan

On Sun, Mar 1, 2020 at 11:52 PM 'Orion Hodson' via Android Building <[hidden email]> wrote:
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <[hidden email]> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.


I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/6062ea95-ea49-483d-8e32-675bbf2a954en%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/CALQgHdkVtH9xh7vgRFGnasMae%2BOeVQt72ab_TunbSe6YFXoOow%40mail.gmail.com.
Reply | Threaded
Open this post in threaded view
|

Re: how to distribute android 10/11 ?

Eli Chang
In reply to this post by Dan Willemsen-2
Hi Dan,

I'm very interested in RBE, but no TAM. Could you tell me the specific method of building RBE on AOSP? 

Is RBE part of Bazel? Is there any document for reference?

thanks

在2020年3月4日星期三 UTC+8 上午5:57:32<Dan Willemsen> 写道:
The build team does not recommend using Goma -- it does not work particularly well with the Android build, and we expect to be removing support for it in master in the coming months (support will most likely still be in this years release, but not next years).

RBE is the replacement, but it's still evolving at the moment. Many of our CI builds have switched to it (they never used Goma), but only a handful of developers have used it so far. If you're interested in this, I'd highly recommend asking your TAM about it (or if you don't have one, contact me and I can try to route your request to the right team).

For what we're doing (other than Goma/RBE), the majority of our CI builds are done on n1-standard-64 machines with SSD persistent disks. Depending on your limitations/requirements, the newer C2/N2 machine families may be more efficient. For local developers, we're generally buying fast desktops -- 2x Intel Xeon GOLD 6154 processor (18-core Skylake, up to 3.70 GHz) with 192 GB RAM and a few TB of SSDs.

- Dan

On Sun, Mar 1, 2020 at 11:52 PM 'Orion Hodson' via Android Building <[hidden email]> wrote:
Hi Alexander

Internally Android developers use goma. The thread below has pointers to the client and server. It is built on GCP.

Kind regards
Orion

On Sun, 1 Mar 2020 at 22:39, alexander zhu <[hidden email]> wrote:
Hi,

I faced one problem, i have tried to solve it many days.


AOSP compile on one powerful physical server(36U128G) is fast . Because my company has so many people need to compile AOSP on the server,so it is busy all the time , and I do not have money to buy more.But the VM I can request more.So I want to use VM on the cloud to compile AOSP.

I have some distribute compile experiences in other c/c++ projects. So I'm thinking is there any way to compile AOSP in distribute way?exp: use many VMs for one AOSP build.


I tried distcc , but failed . AOSP switching to clang, but distcc support clang not ok.https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=926884


So, I want to know ,AOSP in google , how to solve the long build time ?use more powerful physical server? or you have your own distribute compile solution?

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/e3dd71e8-e452-48e2-8686-c51e359cbf5c%40googlegroups.com.

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].

--
--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to [hidden email]
To unsubscribe from this group, send email to
[hidden email]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

---
You received this message because you are subscribed to the Google Groups "Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/android-building/fcb564b9-e266-4789-a2d7-4f2d35f20b5an%40googlegroups.com.