Hi Maxim,
To return 2MB we need to change the logic in huge_page_size()
(odP-system_info.c)function as below.
Do not know about old logic.
I do not see huge page allocation error if I change the logic as below.
diff --git a/platform/linux-generic/odp_system_info.c
b/platform/linux-generic/odp_system_info.c
index 0f1f3c7..2173763 100644
--- a/platform/linux-generic/odp_system_info.c
+++ b/platform/linux-generic/odp_system_info.c
@@ -95,7 +95,7 @@ static int huge_page_size(void)
if (sscanf(dirent->d_name, "hugepages-%i", &temp) != 1)
continue;
- if (temp > size)
+ if (temp < size)
size = temp;
}
On Tue, May 10, 2016 at 6:38 PM, Maxim Uvarov <[email protected]>
wrote:
> On 05/10/16 15:53, nousi wrote:
>
>> value is zero in all the huge page parameters.
>>
>>
> That is not the problem. Problem is that we have call:
>
> /**
> * Huge page size in bytes
> *
> * @return Huge page size in bytes
> */
> uint64_t odp_sys_huge_page_size(void);
>
> which worked well for only one huge page sizes.
>
>
> platform/linux-generic/odp_system_info.c
> static int huge_page_size(void)
> {
> if (sscanf(dirent->d_name, "hugepages-%i", &temp) != 1)
> continue;
>
>
> So because 1 GB pages in /proc lists first now then
> odp_sys_huge_page_size() returns 1 GB, not 2 MB pages.
>
> In general we need to think how to fix it cleanly. Probably modify api of
> odp_sys_huge_page_size() to return
> all available huge page sizes.
>
> So it looks like a bug, thank you for taking attention about it.
>
> Maxim.
>
>
>
> root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# ls -l
>> total 0
>> -r--r--r-- 1 root root 4096 May 10 18:15 free_hugepages
>> -rw-r--r-- 1 root root 4096 May 10 18:15 nr_hugepages
>> -rw-r--r-- 1 root root 4096 May 10 18:15 nr_hugepages_mempolicy
>> -rw-r--r-- 1 root root 4096 May 10 18:15 nr_overcommit_hugepages
>> -r--r--r-- 1 root root 4096 May 10 18:15 resv_hugepages
>> -r--r--r-- 1 root root 4096 May 10 18:15 surplus_hugepages
>> root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat
>> free_hugepages
>> 0
>> root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat
>> nr_hugepages
>> 0
>> root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat
>> nr_hugepages_mempolicy
>> 0
>> root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat
>> nr_overcommit_hugepages
>> 0
>> root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat
>> resv_hugepages
>> 0
>> root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB# cat
>> surplus_hugepages
>> 0
>> root@ubuntu-15-10:/sys/kernel/mm/hugepages/hugepages-1048576kB#
>>
>> On Tue, May 10, 2016 at 6:19 PM, nousi <[email protected] <mailto:
>> [email protected]>> wrote:
>>
>> yes, there are two directories under "/sys/kernel/mm/hugepages"
>>
>> root@ubuntu-15-10:/sys/kernel/mm/hugepages# ls -l
>> total 0
>> drwxr-xr-x 2 root root 0 May 10 18:15 hugepages-1048576kB
>> drwxr-xr-x 2 root root 0 May 10 18:15 hugepages-2048kB
>>
>>
>> On Tue, May 10, 2016 at 6:13 PM, Maxim Uvarov
>> <[email protected] <mailto:[email protected]>> wrote:
>>
>> In my case there is 1 Gb huge page in:
>> /sys/kernel/mm/hugepages
>>
>> and odp_shm_reserve() just rounds all allocations to 1 GB:
>>
>> #ifdef MAP_HUGETLB
>> huge_sz = odp_sys_huge_page_size(); <--- 1 GB here
>> need_huge_page = (huge_sz && alloc_size > page_sz);
>> /* munmap for huge pages requires sizes round up by page */
>> alloc_hp_size = (size + align + (huge_sz - 1)) & (-huge_sz);
>> #endif
>>
>> Do you see the same thing?
>>
>> Maxim.
>>
>> On 05/10/16 15:07, nousi wrote:
>>
>> Hi Maxim,
>>
>> Thanks for your support.
>>
>> mamp return below error;
>> "mmap: Cannot allocate memory"
>> "mount -t hugetlbfs none /mnt/hugetlbfs" also does not help.
>>
>> Huge page allocation success in below two calls after that
>> it fails.
>> 1) odp_thread_globals
>> 2) odp_buffer_pools
>>
>> Please have a look at the console logs below.
>>
>> odp_shared_memory.c:299:odp_shm_reserve():
>> odp_thread_globals: huge page allocation success !
>> odp_shared_memory.c:299:odp_shm_reserve():
>> odp_buffer_pools: huge page allocation success !
>> odp_pool.c:104:odp_pool_init_global():
>> Pool init global
>> odp_pool.c:105:odp_pool_init_global(): pool_entry_s size
>> 8512
>> odp_pool.c:106:odp_pool_init_global(): pool_entry_t size
>> 8512
>> odp_pool.c:107:odp_pool_init_global(): odp_buffer_hdr_t
>> size 216
>> odp_pool.c:108:odp_pool_init_global():
>> *mmap: Cannot allocate memory*
>> odp_queue.c:130:odp_queue_init_global():Queue init ...
>> odp_shared_memory.c:297:odp_shm_reserve(): odp_queues:
>> No huge pages, fall back to normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>>
>>
>> Recently I Moved from Ubuntu 14.04 to 15.10, in Ubuntu
>> 14.04 I had run calssfier example without any huge page
>> allocation error.
>>
>> On Tue, May 10, 2016 at 3:50 PM, Maxim Uvarov
>> <[email protected] <mailto:[email protected]>
>> <mailto:[email protected]
>> <mailto:[email protected]>>> wrote:
>>
>> Does:
>> mount -t hugetlbfs none /mnt/hugetlbfs
>>
>> help?
>>
>> Maxim.
>>
>>
>> On 10 May 2016 at 13:16, Maxim Uvarov
>> <[email protected] <mailto:[email protected]>
>> <mailto:[email protected]
>> <mailto:[email protected]>>> wrote:
>>
>> looks like you have enough free HP. Which error
>> returns
>> mmap() on try with HP?
>>
>> On 10 May 2016 at 11:57, nousi
>> <[email protected] <mailto:[email protected]>
>> <mailto:[email protected]
>>
>> <mailto:[email protected]>>> wrote:
>>
>> linaro@ubuntu-15-10:~$ cat
>> /proc/sys/vm/nr_hugepages
>> 1024
>> linaro@ubuntu-15-10:~$
>>
>> linaro@ubuntu-15-10:~$ cat /proc/meminfo
>> MemTotal: 8061836 kB
>> MemFree: 470516 kB
>> MemAvailable: 1901932 kB
>> Buffers: 92600 kB
>> Cached: 1939696 kB
>> SwapCached: 7516 kB
>> Active: 3238960 kB
>> Inactive: 1804492 kB
>> Active(anon): 2712440 kB
>> Inactive(anon): 1069492 kB
>> Active(file): 526520 kB
>> Inactive(file): 735000 kB
>> Unevictable: 76 kB
>> Mlocked: 76 kB
>> SwapTotal: 16547836 kB
>> SwapFree: 16370160 kB
>> Dirty: 19816 kB
>> Writeback: 0 kB
>> AnonPages: 3004784 kB
>> Mapped: 679960 kB
>> Shmem: 770776 kB
>> Slab: 264692 kB
>> SReclaimable: 212172 kB
>> SUnreclaim: 52520 kB
>> KernelStack: 11952 kB
>> PageTables: 65780 kB
>> NFS_Unstable: 0 kB
>> Bounce: 0 kB
>> WritebackTmp: 0 kB
>> CommitLimit: 19530176 kB
>> Committed_AS: 11165432 kB
>> VmallocTotal: 34359738367 kB
>> VmallocUsed: 410416 kB
>> VmallocChunk: 34358947836 kB
>> HardwareCorrupted: 0 kB
>> AnonHugePages: 583680 kB
>> CmaTotal: 0 kB
>> CmaFree: 0 kB
>> HugePages_Total: 1024
>> HugePages_Free: 1022
>> HugePages_Rsvd: 1022
>> HugePages_Surp: 0
>> Hugepagesize: 2048 kB
>> DirectMap4k: 234400 kB
>> DirectMap2M: 6991872 kB
>> DirectMap1G: 2097152 kB
>> linaro@ubuntu-15-10:~$
>>
>>
>> On Tue, May 10, 2016 at 12:38 PM, Maxim Uvarov
>> <[email protected]
>> <mailto:[email protected]>
>> <mailto:[email protected]
>> <mailto:[email protected]>>>
>> wrote:
>>
>> Please put output for:
>> cat /proc/meminfo
>> cat /proc/sys/vm/nr_hugepages
>>
>> Thank you,
>> Maxim.
>>
>>
>>
>> On 10 May 2016 at 08:36, nousi
>> <[email protected] <mailto:[email protected]>
>> <mailto:[email protected]
>> <mailto:[email protected]>>> wrote:
>>
>> mmap is failing in "odp_shm_reserve"
>> function
>> (odp_queue_init_global() --->
>> odp_shm_reserve()
>> ---> odp_shm_reserve())
>>
>>
>> debug logs:
>>
>> root@ubuntu-15-10:/home/linaro/linaro/odp/example/classifier#
>> ./odp_classifier -i eno1 -m 0 -p
>> "ODP_PMR_SIP_ADDR:192.168.10.11
>> <tel:192.168.10.11>:FFFFFFFF:queue1"
>> -p "ODP_PMR_SIP_ADDR:10.130.69.0
>> <http://10.130.69.0>:000000FF:queue2"
>> -p "ODP_PMR_SIP_ADDR:10.130.68.0
>> <http://10.130.68.0>:FFFFFE00:queue3"
>>
>> odp_pool.c:104:odp_pool_init_global():
>> Pool init global
>> odp_pool.c:105:odp_pool_init_global():
>> pool_entry_s size 8512
>> odp_pool.c:106:odp_pool_init_global():
>> pool_entry_t size 8512
>> odp_pool.c:107:odp_pool_init_global():
>> odp_buffer_hdr_t size 216
>> odp_pool.c:108:odp_pool_init_global():
>> odp_queue.c:130:odp_queue_init_global():Queue init
>> ...
>> odp_shared_memory.c:296:odp_shm_reserve():odp_queues:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_queue.c:154:odp_queue_init_global():done
>> odp_queue.c:155:odp_queue_init_global():Queue init
>> global
>> odp_queue.c:157:odp_queue_init_global(): struct
>> queue_entry_s size 320
>> odp_queue.c:159:odp_queue_init_global():
>> queue_entry_t size 320
>> odp_queue.c:160:odp_queue_init_global():
>> odp_schedule.c:145:odp_schedule_init_global():Schedule
>> init ...
>> odp_shared_memory.c:296:odp_shm_reserve():odp_scheduler:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_shared_memory.c:296:odp_shm_reserve():odp_sched_pool:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_schedule.c:226:odp_schedule_init_global():done
>> odp_shared_memory.c:296:odp_shm_reserve():odp_pktio_entries:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_shared_memory.c:296:odp_shm_reserve():crypto_pool:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_shared_memory.c:296:odp_shm_reserve():shm_odp_cos_tbl:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_shared_memory.c:296:odp_shm_reserve():shm_odp_pmr_tbl:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> main :: odp_init_global done!
>> odp_classifier.c:500:main():main :: odp_init_local
>> done!
>> odp_classifier.c:505:main():main ::
>> odp_shm_reserve done!
>>
>> ODP system info
>> ---------------
>> ODP API version: 1.10.0
>> CPU model: Intel(R) Core(TM) i7-5600U CPU
>> CPU freq (hz): 2600000000
>> Cache line size: 64
>> CPU count: 4
>>
>> Running ODP appl: "odp_classifier"
>> -----------------
>> Using IF:eno1
>>
>> num worker threads: 2
>> first CPU: 2
>> cpu mask: 0xC
>> odp_shared_memory.c:296:odp_shm_reserve():packet_pool:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_pool.c:759:odp_pool_print():Pool info
>> odp_pool.c:760:odp_pool_print():---------
>> odp_pool.c:762:odp_pool_print(): pool 1
>> odp_pool.c:764:odp_pool_print(): name packet_pool
>> odp_pool.c:769:odp_pool_print(): pool type packet
>> odp_pool.c:771:odp_pool_print(): pool storage ODP
>> managed shm handle 11
>> odp_pool.c:773:odp_pool_print(): pool status active
>> odp_pool.c:777:odp_pool_print(): pool opts
>> segmented, non-zeroized, created
>> odp_pool.c:778:odp_pool_print(): pool base
>> 0x7f5091aab000
>> odp_pool.c:780:odp_pool_print(): pool size 1310720
>> (320 pages)
>> odp_pool.c:781:odp_pool_print(): pool mdata base
>> 0x7f5091bb5940
>> odp_pool.c:782:odp_pool_print(): udata size 0
>> odp_pool.c:783:odp_pool_print(): headroom 66
>> odp_pool.c:784:odp_pool_print(): tailroom 0
>> odp_pool.c:791:odp_pool_print(): seg length 1856
>> requested, 1936 used
>> odp_pool.c:793:odp_pool_print(): pkt length 1856
>> requested, 1936 used
>> odp_pool.c:795:odp_pool_print(): num bufs 564
>> odp_pool.c:797:odp_pool_print(): bufs available 564
>> odp_pool.c:798:odp_pool_print(): bufs in use 0
>> odp_pool.c:799:odp_pool_print(): buf allocs 0
>> odp_pool.c:800:odp_pool_print(): buf frees 0
>> odp_pool.c:801:odp_pool_print(): buf empty 0
>> odp_pool.c:803:odp_pool_print(): blk size 1936
>> odp_pool.c:805:odp_pool_print(): blks available 564
>> odp_pool.c:806:odp_pool_print(): blk allocs 0
>> odp_pool.c:807:odp_pool_print(): blk frees 0
>> odp_pool.c:808:odp_pool_print(): blk empty 0
>> odp_pool.c:809:odp_pool_print(): buf high wm
>> value 282
>> odp_pool.c:810:odp_pool_print(): buf high wm count 0
>> odp_pool.c:811:odp_pool_print(): buf low wm
>> value 141
>> odp_pool.c:812:odp_pool_print(): buf low wm count 0
>> odp_pool.c:813:odp_pool_print(): blk high wm
>> value 282
>> odp_pool.c:814:odp_pool_print(): blk high wm count 0
>> odp_pool.c:815:odp_pool_print(): blk low wm
>> value 141
>> odp_pool.c:816:odp_pool_print(): blk low wm count 0
>> main :: odp_pool_print done!
>> odp_packet_io.c:230:setup_pktio_entry():eno1 uses
>> socket_mmap
>> created pktio:01, dev:eno1, queue
>> mode (ATOMIC
>> queues)
>> default pktio01
>> odp_shared_memory.c:296:odp_shm_reserve():
>> DefaultPool:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_shared_memory.c:296:odp_shm_reserve():
>> queue1Pool0:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_shared_memory.c:296:odp_shm_reserve():
>> queue2Pool1:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>> odp_shared_memory.c:296:odp_shm_reserve():
>> queue3Pool2:
>> :: No huge pages, fall back to
>> normal pages,
>> check: /proc/sys/vm/nr_hugepages.
>>
>> ----------------------------------------
>> CLASSIFIER EXAMPLE STATISTICS
>> ----------------------------------------
>> CONFIGURATION
>>
>> COS VALUE MASK
>> ----------------------------------------
>> queue1 192.168.10.11
>> <tel:192.168.10.11> FFFFFFFF
>> queue2 10.130.69.0 000000FF
>> queue3 10.130.68.0 FFFFFE00
>>
>> RECEIVED PACKETS
>> ----------------------------------------
>> queue1 |queue2 |queue3 |DefaultCos
>> |Total Packets
>> queue pool |queue pool |queue pool
>> |queue pool |
>> 845 845 |0 0 |0 0 |38
>> 38 |883
>>
>>
>> On Mon, May 9, 2016 at 9:00 PM, Bill
>> Fischofer
>> <[email protected]
>> <mailto:[email protected]>
>> <mailto:[email protected]
>> <mailto:[email protected]>>> wrote:
>>
>>
>>
>> On Mon, May 9, 2016 at 6:57 AM, nousi
>> <[email protected]
>> <mailto:[email protected]>
>> <mailto:[email protected]
>> <mailto:[email protected]>>> wrote:
>>
>>
>> Hi All,
>>
>> Pleas help me in running ODP
>> classifier
>> example with huge pages.
>> In ubuntu 15.10 by default
>> interface
>> naming as "eno1" and value in
>> "/proc/sys/vm/nr_hugepages." is 1024.
>> classifier example program
>> could not able
>> to use huge pages even though
>> nr_hugepages
>> value is non zero.
>> I could able to run the
>> classier example,
>> but it is not using huge pages.
>> /*
>> console log is pasted below
>> for you reference.
>>
>> */root@odp/example/classifier#
>>
>> ./odp_classifier -i eno1 -m 0 -p
>> "ODP_PMR_SIP_ADDR:192.168.10.11
>> <tel:192.168.10.11>:FFFFFFFF:queue1"
>> -p
>> "ODP_PMR_SIP_ADDR:10.130.69.0
>> <http://10.130.69.0>:000000FF:queue2"
>> -p
>> "ODP_PMR_SIP_ADDR:10.130.68.0
>> <http://10.130.68.0>:FFFFFE00:queue3"
>>
>> odp_pool.c:104:odp_pool_init_global():
>> Pool init global
>> odp_pool.c:105:odp_pool_init_global():
>> pool_entry_s size 8512
>> odp_pool.c:106:odp_pool_init_global():
>> pool_entry_t size 8512
>> odp_pool.c:107:odp_pool_init_global():
>> odp_buffer_hdr_t size 216
>> odp_pool.c:108:odp_pool_init_global():
>> odp_queue.c:130:odp_queue_init_global():Queue
>> init ...
>> odp_shared_memory.c:296:odp_shm_reserve():
>> odp_queues:
>> No huge pages, fall back
>> to normal pages,
>> check:
>> /proc/sys/vm/nr_hugepages.
>>
>>
>> This is an informational message
>> saying that
>> the linux-generic implementation
>> was unable to
>> allocate huge pages so it's
>> falling back to
>> normal pages. I'm not sure why
>> you're seeing
>> that except that it seems that some
>> allocations may have been
>> successful (those in
>> odp_pool.c) while those for queue
>> initialization were not.
>>
>> I'll let others who are more
>> expert in this
>> area chime in with some additional
>> thoughts.
>>
>>
>>
>>
>> Thanks & Regards,
>> /B.Nousilal,
>> /
>>
>> _______________________________________________
>> lng-odp mailing list
>> [email protected] <mailto:[email protected]>
>> <mailto:
>> [email protected]
>> <mailto:[email protected]>>
>> https://lists.linaro.org/mailman/listinfo/lng-odp
>>
>>
>>
>>
>>
>> -- /Thanks & Regards,
>> B.Nousilal,
>> /
>>
>> _______________________________________________
>> lng-odp mailing list
>> [email protected] <mailto:[email protected]>
>> <mailto:[email protected]
>> <mailto:[email protected]>>
>> https://lists.linaro.org/mailman/listinfo/lng-odp
>>
>>
>>
>>
>>
>> -- /Thanks & Regards,
>> B.Nousilal,
>> /
>>
>>
>>
>>
>>
>>
>> -- /Thanks & Regards,
>> B.Nousilal,
>> /
>>
>>
>>
>>
>>
>> -- /Thanks & Regards,
>> B.Nousilal,
>> /
>>
>>
>>
>>
>> --
>> /Thanks & Regards,
>> B.Nousilal,
>> /
>>
>
>
--
*Thanks & Regards,B.Nousilal,*
_______________________________________________
lng-odp mailing list
[email protected]
https://lists.linaro.org/mailman/listinfo/lng-odp
