Thank you for all your help and good luck with improving finder! With
regards to memory, I have a measly 4GB of ram so that is probally slowing
finder down and I do expect it to be slow (I do plan on upgrading in case
you are wondering). However, I do not believe what I am experiencing is the
same as what Alessandro is experiencing. Again, Thanks for all your help!
On Thursday, December 21, 2017 at 8:14:39 PM UTC, Colin Cross wrote:
>
> Yeah, this is showing very slow lstat calls:
> [pid 4007] <... lstat resumed> {st_mode=S_IFDIR|0777, st_size=4096, ...})
> = 0 <0.020417>
> vs. mine:
> [pid 219359] <... lstat resumed> {st_mode=S_IFREG|0640, st_size=323161,
> ...}) = 0 <0.000046>
>
> I don't know why yours would be so much slower, maybe just round trip
> latency to the external drive and not enough memory to keep them all in
> the inode cache?
>
> We can improve finder to drastically reduce the number of lstat calls, but
> ninja is going to have to stat most of the files to determine what needs
> building so I don't know how much improvement you will see.
>
>
> On Thu, Dec 21, 2017 at 11:56 AM, Peter Warrington <[email protected]
> <javascript:>> wrote:
>
>> Sorry about that. Is this attached strace any better?
>>
>> On Thursday, December 21, 2017 at 7:36:22 PM UTC, Colin Cross wrote:
>>>
>>> I don't see the data I'd expect with -T, it should have a time in angle
>>> brackets after each finished syscall:
>>> [pid 219359] <... lstat resumed> {st_mode=S_IFREG|0640, st_size=323161,
>>> ...}) = 0 <0.000046>
>>>
>>>
>>> On Thu, Dec 21, 2017 at 3:05 AM, Peter Warrington <[email protected]>
>>> wrote:
>>>
>>>> Attached is a much longer strace with the -T option used.
>>>>
>>>> On Wednesday, December 20, 2017 at 11:16:53 PM UTC, Colin Cross wrote:
>>>>>
>>>>> The results of the finder are cached in out/.module_paths/files.db, so
>>>>> once the command has completed successfully once the next runs will be
>>>>> much
>>>>> faster. You could try wiping that file to reproduce it again, or run
>>>>> with
>>>>> a different value for OUT_DIR (OUT_DIR=out2 lunch ...).
>>>>>
>>>>> On Wed, Dec 20, 2017 at 1:58 PM, Peter Warrington <[email protected]
>>>>> > wrote:
>>>>>
>>>>>> Hello, the command has now started to work like normal after leaving
>>>>>> it for a while, so my problem might not be the same as Alessandro's.
>>>>>> Unfortunately, I cannot upload a bigger stack trace until tomorrow
>>>>>> because
>>>>>> of the time in my timezone. Sorry!
>>>>>>
>>>>>> On Wednesday, December 20, 2017 at 9:48:34 PM UTC, Colin Cross wrote:
>>>>>>>
>>>>>>> Can you attach a longer strace with -T ?
>>>>>>>
>>>>>>> On Wed, Dec 20, 2017 at 1:28 PM, Peter Warrington <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> Thank you! That is very interesting. In case it helps, I
>>>>>>>> benchmarked my external HDD and it has an average read rate of 92.0
>>>>>>>> MB/s
>>>>>>>> and an average access time of 16.37 msec. However, I didn't have any
>>>>>>>> trouble with this previously.
>>>>>>>>
>>>>>>>> On Wednesday, December 20, 2017 at 9:06:16 PM UTC, Colin Cross
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Nothing looks wrong in that strace, it is making forward
>>>>>>>>> progress. It needs to read every directory and stat every file in
>>>>>>>>> the
>>>>>>>>> tree, is your filesystem very slow for some reason?
>>>>>>>>>
>>>>>>>>> One difference I see between the old implementation using the find
>>>>>>>>> tool and the new implementation is that find seems to be able to
>>>>>>>>> avoiding
>>>>>>>>> calling stat on every directory entry, because the relevant
>>>>>>>>> information
>>>>>>>>> (whether the entry is a subdirectory or a file) is contained in the
>>>>>>>>> return
>>>>>>>>> from the getdents64 syscall. The Go implementation is ignoring that
>>>>>>>>> information and has to call lstat to get it.
>>>>>>>>>
>>>>>>>>> On Wed, Dec 20, 2017 at 11:32 AM, Peter Warrington <
>>>>>>>>> [email protected]> wrote:
>>>>>>>>>
>>>>>>>>>> Attached is a partial strace.
>>>>>>>>>>
>>>>>>>>>> On Wednesday, December 20, 2017 at 7:06:31 PM UTC, Colin Cross
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> From the stack trace after ctrl-c, lunch will be waiting for the
>>>>>>>>>>> child soong_ui process to complete. An strace of that process
>>>>>>>>>>> would be
>>>>>>>>>>> interesting.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Dec 18, 2017 at 4:12 PM, Alessandro Brucato <
>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> This is the output of strace attached to the process in which I
>>>>>>>>>>>> execute "lunch".
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <https://lh3.googleusercontent.com/-Rk-T1kCPUnA/WjhYORJYCKI/AAAAAAAAGSY/LIzqOc58jvkTpj7dowKqjz1OsKoAy0dZQCLcBGAs/s1600/strace.PNG>
>>>>>>>>>>>>
>>>>>>>>>>>> When "lunch" is stuck, the output of strace ends with
>>>>>>>>>>>> "wait4(-1,". All the remaining part appears when I abort "lunch"
>>>>>>>>>>>> with
>>>>>>>>>>>> CTRL+C.
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe this issue is due to the fact that I'm using a guest in
>>>>>>>>>>>> Virtualbox?
>>>>>>>>>>>>
>>>>>>>>>>>> Il giorno lunedì 18 dicembre 2017 23:45:48 UTC+1, Colin Cross
>>>>>>>>>>>> ha scritto:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can you capture the full output of all the stack traces?
>>>>>>>>>>>>>
>>>>>>>>>>>>> It's getting stuck while trying to walk all the checked-out
>>>>>>>>>>>>> code looking for various build files. If it was completely
>>>>>>>>>>>>> deadlocked then
>>>>>>>>>>>>> it would have died on its own, so it's probably stuck in a loop
>>>>>>>>>>>>> somehow.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Dec 18, 2017 at 8:12 AM, Alessandro Brucato <
>>>>>>>>>>>>> [email protected]> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello, I'm trying to build the Android source code for my
>>>>>>>>>>>>>> Hikey 960 board.
>>>>>>>>>>>>>> I'm following this video tutorial
>>>>>>>>>>>>>> https://www.youtube.com/watch?v=RD_NZRKiZBU in parallel to
>>>>>>>>>>>>>> the official Android guide
>>>>>>>>>>>>>> https://source.android.com/setup/devices#hikey-boards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Everything went fine until I reached the command "lunch
>>>>>>>>>>>>>> hikey960-userdebug". It blocks, without returning any output. I
>>>>>>>>>>>>>> left it
>>>>>>>>>>>>>> "working" for 1 hour and then I aborted it (even if I know that
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>> command should take few seconds). If I execute only "lunch" it
>>>>>>>>>>>>>> shows the
>>>>>>>>>>>>>> list of available build targets, and when I select
>>>>>>>>>>>>>> "hikey960-userdebug" it
>>>>>>>>>>>>>> blocks. When I abort it with CTRL+C, it says "Still alive,
>>>>>>>>>>>>>> killing
>>>>>>>>>>>>>> subprocesses" and then it shows a lot of messages referring to
>>>>>>>>>>>>>> threads.
>>>>>>>>>>>>>> Here is the screenshot https://ibb.co/k32KQR
>>>>>>>>>>>>>> I noticed that also the "make" command (inside the AOSP
>>>>>>>>>>>>>> directory) has the same behaviour.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is the list of commands I've used during the process:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> sudo apt-get install openjdk-7-jdk
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> sudo apt-get install git-core gnupg flex bison gperf
>>>>>>>>>>>>>> build-essential zip curl zlib1g-dev gcc-multilib g++-multilib
>>>>>>>>>>>>>> libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev
>>>>>>>>>>>>>> lib32z-dev
>>>>>>>>>>>>>> ccache libgl1-mesa-dev libxml2-utils xsltproc unzip
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> mkdir output
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> export OUT_DIR_COMMON_BASE=/home/brucedh/output/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> mkdir ~/bin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> PATH=~/bin:$PATH
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> curl https://storage.googleapis.com/git-repo-downloads/repo
>>>>>>>>>>>>>> > ~/bin/repo
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> chmod a+x ~/bin/repo
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> mkdir dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> cd dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> mkdir hikey960
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> cd hikey960
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> git config --global user.name "<REDACTED>"
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> git config --global user.email "<REDACTED>"
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> repo init -u
>>>>>>>>>>>>>> https://android.googlesource.com/platform/manifest -b master
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> repo sync –j4
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> git clone
>>>>>>>>>>>>>> https://android.googlesource.com/kernel/hikey-linaro
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> cd hikey-linaro
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> git checkout -b android-hikey-linaro-4.9
>>>>>>>>>>>>>> origin/android-hikey-linaro-4.9
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> make ARCH=arm64 menuconfig
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> make ARCH=arm64 hikey960_defconfig
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> PATH=/home/brucedh/dev/hikey960/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/:$PATH
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> make ARCH=arm64 CROSS_COMPILE=aarch64-linux-android- -j4
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> cp arch/arm64/boot/dts/hisilicon/hi3660-hikey960.dtb
>>>>>>>>>>>>>> ../device/linaro/hikey-kernel/hi3660-hikey960.dtb-4.9
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> cp arch/arm64/boot/Image.gz
>>>>>>>>>>>>>> ../device/linaro/hikey-kernel/Image.gz-hikey960-4.9
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> cd ~/dev/hikey960
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> . ./build/envsetup.sh
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> lunch hikey960-userdebug
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm using Ubuntu 16.04 LTS as guest on Virtualbox. I tried to
>>>>>>>>>>>>>> do all the process once again from the beginning with a new
>>>>>>>>>>>>>> virtualbox
>>>>>>>>>>>>>> image, but I found the same problem.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks in advance!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>>> "Android Building" mailing list.
>>>>>>>>>>>>>> To post to this group, send email to
>>>>>>>>>>>>>> [email protected]
>>>>>>>>>>>>>> To unsubscribe from this group, send email to
>>>>>>>>>>>>>> [email protected]
>>>>>>>>>>>>>> For more options, visit this group at
>>>>>>>>>>>>>> http://groups.google.com/group/android-building?hl=en
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>>>> Google Groups "Android Building" group.
>>>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>>>> it, send an email to [email protected].
>>>>>>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>> --
>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>> "Android Building" mailing list.
>>>>>>>>>>>> To post to this group, send email to
>>>>>>>>>>>> [email protected]
>>>>>>>>>>>> To unsubscribe from this group, send email to
>>>>>>>>>>>> [email protected]
>>>>>>>>>>>> For more options, visit this group at
>>>>>>>>>>>> http://groups.google.com/group/android-building?hl=en
>>>>>>>>>>>>
>>>>>>>>>>>> ---
>>>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>>>> Google Groups "Android Building" group.
>>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from
>>>>>>>>>>>> it, send an email to [email protected].
>>>>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>> --
>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>> "Android Building" mailing list.
>>>>>>>>>> To post to this group, send email to [email protected]
>>>>>>>>>> To unsubscribe from this group, send email to
>>>>>>>>>> [email protected]
>>>>>>>>>> For more options, visit this group at
>>>>>>>>>> http://groups.google.com/group/android-building?hl=en
>>>>>>>>>>
>>>>>>>>>> ---
>>>>>>>>>> You received this message because you are subscribed to the
>>>>>>>>>> Google Groups "Android Building" group.
>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>>> send an email to [email protected].
>>>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>> --
>>>>>>>> You received this message because you are subscribed to the
>>>>>>>> "Android Building" mailing list.
>>>>>>>> To post to this group, send email to [email protected]
>>>>>>>> To unsubscribe from this group, send email to
>>>>>>>> [email protected]
>>>>>>>> For more options, visit this group at
>>>>>>>> http://groups.google.com/group/android-building?hl=en
>>>>>>>>
>>>>>>>> ---
>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>> Groups "Android Building" group.
>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>> send an email to [email protected].
>>>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>> --
>>>>>> You received this message because you are subscribed to the "Android
>>>>>> Building" mailing list.
>>>>>> To post to this group, send email to [email protected]
>>>>>> To unsubscribe from this group, send email to
>>>>>> [email protected]
>>>>>> For more options, visit this group at
>>>>>> http://groups.google.com/group/android-building?hl=en
>>>>>>
>>>>>> ---
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "Android Building" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>>
>>>>>
>>>>> --
>>>> --
>>>> You received this message because you are subscribed to the "Android
>>>> Building" mailing list.
>>>> To post to this group, send email to [email protected]
>>>> To unsubscribe from this group, send email to
>>>> [email protected]
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/android-building?hl=en
>>>>
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Android Building" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>> --
>> --
>> You received this message because you are subscribed to the "Android
>> Building" mailing list.
>> To post to this group, send email to [email protected]
>> <javascript:>
>> To unsubscribe from this group, send email to
>> [email protected] <javascript:>
>> For more options, visit this group at
>> http://groups.google.com/group/android-building?hl=en
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Android Building" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
--
You received this message because you are subscribed to the "Android Building"
mailing list.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en
---
You received this message because you are subscribed to the Google Groups
"Android Building" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
