Mark, I have submitted PR#202 for all of my code changes and additions.
Subutai, that is what I thought. Thank you for the clarification.
When I run htmtest through gdb I get the following output:
(gdb) handle SIGILL nostop
Signal Stop Print Pass to program Description
SIGILL No Yes Yes Illegal instruction
(gdb) r
Starting program: /home/pi/nta/eng/bin/htmtest
<snip>
Adding a PyNode region...
Program received signal SIGSEGV, Segmentation fault.
nta::Link::initialize (this=0x1, destinationOffset=1600704) at
/home/pi/nupic/nta/engine/Link.cpp:139
139 if(dest_->isRegionLevel())
(gdb) where
#0 nta::Link::initialize (this=0x1, destinationOffset=1600704) at
/home/pi/nupic/nta/engine/Link.cpp:139
#1 0x00187c20 in ?? ()
#2 0x00187c20 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb) list
134 else
135 {
136 NTA_CHECK(srcD.isDontcare() || srcD ==
src_->getRegion().getDimensions());
137 }
138
139 if(dest_->isRegionLevel())
140 {
141 Dimensions d;
142 for(size_t i = 0; i < dest_->getRegion().getDimensions().size();
i++)
143 {
(gdb)
Any ideas why the isRegionLevel() would cause a SEGV?
Thanks,
Matt
On Aug 22, 2013, at 4:35 AM, Marek Otahal <[email protected]> wrote:
> Great work Matt! Thanks.
> Can you make a PR and get this stuff upstream once you think the branch is
> ready?
> thank you, Mark
>
>
> On Thu, Aug 22, 2013 at 6:08 AM, Subutai Ahmad <[email protected]> wrote:
>
> That's excellent!
>
> You can safely comment out the use of getticks. It's a counter that counts
> the number of CPU cycles. We used it for timing small bits of code that are
> called often and you need a very low overhead timing call. Not really sure
> why those unused references were left in! I think those calls were inserted
> manually when someone was actively working on optimizing that bit of code.
>
> --Subutai
>
>
> On Wed, Aug 21, 2013 at 7:57 PM, Matt Keith <[email protected]> wrote:
> Progress, here is the result of the run_tests script:
> === 216 passed, 4 skipped in 3149.59 seconds ===
>
> No more errors! I am doing a clean build now and will test some more in the
> morning. I have committed all of my files and changes to my keithcom github
> fork.
>
> Does anyone know what the getticks function is used for in the
> cycle_counter.hpp file? I was working on getting that to work on the Pi,
> when I noticed that the only place it was referenced was in the Cells4.cpp
> file. That file calls the function and stores the result in a local variable
> and then never references it anywhere. Is there some funny business going on
> that I am not seeing, or is this just left over from some old code? For now,
> I have just commented out the references in the Cells4.cpp file. Please let
> me know if this will cause a problem somewhere.
>
> Thanks,
>
> Matt
>
> On Aug 21, 2013, at 9:05 AM, Georgios Pierris <[email protected]> wrote:
>
>> Uhmm.. I see. I had some similar issues when I wanted to distribute a
>> virtual machine and had to shrink it. There were a couple of ways to do it,
>> not sure if applicable in your case.
>>
>> One was to use dd and write a massive file full of zeros. That will pretty
>> much take up all the free unallocated space on your sd card which had any
>> values from past write operations. Then you simply delete it. Or you can use
>> a tool called zerofree.
>>
>> After that, you end up with many zeros in your clone, which compression
>> algorithms exploit and you will end up with a much smaller tar.gz or
>> whatever you decide to use.
>>
>> Cheers,
>> Georgios
>>
>>
>> On Wed, Aug 21, 2013 at 3:45 PM, Matt Keith <[email protected]> wrote:
>> Yes, we should be able to just distribute an image file of the full system.
>> I will need to do a clean build as my current system has a bunch of
>> additional items installed as I was investigating the build process. Also,
>> I would like to look into ways to shrink the image as I am currently on an 8
>> GB sd card to allow space for the OS, the 1 GB swap file and software.
>>
>> I'm going to look into my changes in the cycle_counter.hpp file to see if
>> that is causing the seg fault.
>>
>> The quest continues . . .
>>
>> Matt
>>
>> On Aug 21, 2013, at 5:43 AM, Georgios Pierris <[email protected]> wrote:
>>
>>> Great Matt! I am afraid I cannot help you with these errors, but I just
>>> wanted to know, is it safe to assume that once you have compiled everything
>>> you could "distribute" your sd clone with dd so that anyone else could get
>>> started within minutes?
>>>
>>> Cheers,
>>> Georgios
>>>
>>>
>>> On Wed, Aug 21, 2013 at 3:19 AM, Matt Keith <[email protected]> wrote:
>>> Success! Thank you so much Georgios for the helpful information.
>>>
>>> Ok, so the entire build finally completed on the Raspberry Pi after nearly
>>> 6 hours! I had to increase the swap size to 1 GB to get it to complete
>>> without blowing up. Developing on the Pi really brings back memories of
>>> this:
>>> http://imgs.xkcd.com/comics/compiling.png
>>>
>>> That's the good news. Back to reality, it doesn't pass all of the tests.
>>>
>>> The first thing that I noticed is that the
>>> external/linux32/lib/python/site-packages contains some .so files. I was
>>> just using the files from the linux64 dir to try and get things to build,
>>> but we need to get arm versions of the .so files in matplotlib, Crypto, and
>>> PIL. So I used pip to install matplotlib and PIL and copied the dirs from
>>> the system location to the nupic/external dirs. Then I installed
>>> python-crypto with apt-get and copied the Crypto dir over. Is this a valid
>>> way to get the libs? Does anyone know how the shared objects were built
>>> for the current linux64 and darwin64 dirs?
>>>
>>> When I run the nupic/bin/run_tests.py it shows:
>>> === 6 failed, 210 passed, 4 skipped in 275.98 seconds ===
>>>
>>> The htmtest binary dies with a seg fault right after 'Adding a PyNode
>>> region...'. And the testeverything app shows:
>>> Found 40 test sets
>>> WARNING: Caught exception: This exception should get caught.
>>> ======= Tests for TesterTest ==============
>>> Total tests: 10, Failures: 4, Disabled : 0
>>> WARNING: Critical failure ocurred
>>> <snip>
>>> FAIL Integer test, should fail (line: 50)
>>> Expected result: 1
>>> Actual result: 0
>>> FAIL Double test, should fail (line: 51)
>>> Expected result: 23.42
>>> Actual result: 23.421
>>> FAIL String test, should fail (line: 52)
>>> Expected result: Numenta
>>> Actual result: Numenta
>>> <snip>
>>> ERROR: Invalid NTA_BasicType -1 used in array constructor
>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 59]
>>> ERROR: allocateBuffer -- buffer already set. Use releaseBuffer first
>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 80]
>>> ERROR: allocateBuffer -- buffer already set. Use releaseBuffer first
>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 80]
>>> ERROR: setBuffer -- buffer already set. Use releaseBuffer first
>>> [/home/pi/nupic/nta/ntypes/ArrayBase.cpp line 97]
>>> ======= Tests for ArrayTest ==============
>>> Total tests: 72, Failures: 0, Disabled : 0
>>> All tests passed
>>> <snip>
>>> ======= Tests for BasicTypeTest ==============
>>> Total tests: 47, Failures: 0, Disabled : 0
>>> All tests passed
>>> <snip>
>>> ======= Tests for BufferTest ==============
>>> Total tests: 94, Failures: 0, Disabled : 0
>>> All tests passed
>>>
>>> ERROR: CHECK FAILED: "index < vec_.size()"
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70]
>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp line
>>> 95]
>>> ERROR: CHECK FAILED: "index < vec_.size()"
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70]
>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp line
>>> 95]
>>> ERROR: CHECK FAILED: "index < vec_.size()"
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70]
>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp line
>>> 95]
>>> ERROR: CHECK FAILED: "index < vec_.size()"
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 70]
>>> ERROR: No item named: blah [/home/pi/nupic/nta/ntypes/Collection.cpp line
>>> 95]
>>> ERROR: Unable to add item '0' to collection because it already exists
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107]
>>> ERROR: Unable to add item '1' to collection because it already exists
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107]
>>> ERROR: Unable to add item '2' to collection because it already exists
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107]
>>> ERROR: No item named '4' in collection
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127]
>>> ERROR: No item named '1' in collection
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127]
>>> ERROR: Unable to add item '0' to collection because it already exists
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107]
>>> ERROR: Unable to add item '1' to collection because it already exists
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107]
>>> ERROR: Unable to add item '2' to collection because it already exists
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 107]
>>> ERROR: No item named '1' in collection
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127]
>>> ERROR: No item named '0' in collection
>>> [/home/pi/nupic/nta/ntypes/Collection.cpp line 127]
>>> ======= Tests for CollectionTest ==============
>>> Total tests: 199, Failures: 0, Disabled : 0
>>> All tests passed
>>> <snip>
>>> ======= Tests for CondProbTableTest ==============
>>> Total tests: 115, Failures: 0, Disabled : 0
>>> All tests passed
>>> <snip>
>>> ======= Tests for DenseTensorUnitTest ==============
>>> Total tests: 0, Failures: 0, Disabled : 0
>>> All tests passed
>>>
>>> ERROR: Attempt to get count from dimensions [unspecified]
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 57]
>>> ERROR: Bad request for dimension 0 on [unspecified]
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78]
>>> ERROR: Invalid coordinate [1 2] for Dimensions [unspecified]
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 171]
>>> ERROR: Attempt to get count from dimensions [unspecified]
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 57]
>>> ERROR: Bad request for dimension 0 on [unspecified]
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78]
>>> ERROR: Invalid coordinate index 0 of 0 is too large for region dimensions
>>> [dontcare] [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 189]
>>> ERROR: Attempt to get count from dimensions [dontcare]
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 57]
>>> ERROR: Invalid coordinate index 0 of 1 is too large for region dimensions
>>> [1 0] (invalid) [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 189]
>>> ERROR: Attempt to get count from invalid dimensions [1 0] (invalid)
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 62]
>>> ERROR: Bad request for dimension 2 on [1 0] (invalid)
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78]
>>> ERROR: Bad request for dimension 2 on [2 3]
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78]
>>> ERROR: Bad request for dimension 2 on [2 5]
>>> [/home/pi/nupic/nta/ntypes/Dimensions.cpp line 78]
>>> ======= Tests for DimensionsTest ==============
>>> Total tests: 205, Failures: 0, Disabled : 0
>>> All tests passed
>>>
>>> Let me know if you want the full output. Any idea of what piece might be
>>> broken?
>>>
>>> Thanks,
>>>
>>> Matt
>>>
>>> On Aug 19, 2013, at 1:29 PM, Georgios Pierris <[email protected]> wrote:
>>>
>>>> Hello Matt,
>>>>
>>>> have a look on my comment in https://issues.numenta.org/browse/NPC-143
>>>> regarding the redefinition on swig.
>>>>
>>>> Cheers,
>>>> Georgios
>>>>
>>>>
>>>> On Mon, Aug 19, 2013 at 8:25 PM, Matt Keith <[email protected]> wrote:
>>>> Mark,
>>>>
>>>> Your linux32 compiled .a files do not work in the ARM environment, so I
>>>> think that we will need to create a new external/linux32arm dir to support
>>>> that platform. I'll try to get the build working with that change.
>>>>
>>>> I didn't remove any of the grouper files, I just commented out the guts of
>>>> all of the Grouper.cpp methods. However, the problem was with the gcc 4.6
>>>> compiler and not the code, so it just failed on some different files later
>>>> in the build. Upgrading to gcc 4.7 fixed the issue. I'll add my +1 to
>>>> removing the old code, but I don't think that it is an issue for raspberry
>>>> pi anymore.
>>>>
>>>> I spoke a little too soon saying that the build completed in my last
>>>> message. There was an error with apr.h buried in the build output that I
>>>> missed and that is why none of the test scripts were created. I have
>>>> corrected that error but now I am getting the following error with swig.
>>>>
>>>> make[2]: Entering directory
>>>> `/tmp/ntabuild/lang/py/bindings/engine_internal'
>>>> /bin/bash ../../../../libtool --tag=CXX --mode=compile g++
>>>> -DHAVE_CONFIG_H -I. -I/home/pi/nupic/lang/py/bindings/engine_internal
>>>> -I../../../../build_system/unix -I/usr/include/python2.7 -I/home/pi/nupic
>>>> -I/home/pi/nupic/external/linux32/include -DNUPIC2 -I/home/pi/nupic
>>>> -I/home/pi/nupic/external/common/include
>>>> -I/home/pi/nupic/external/linux32/include -fvisibility=hidden
>>>> -DNTA_INTERNAL -g -DNTA_ASSERTIONS_ON -fPIC -DPIC -ffloat-store -D_PY27
>>>> -DNTA_PLATFORM_linux32 -Wall -Wreturn-type -Wunused-variable
>>>> -Wno-deprecated -DBOOST_NO_WREGEX -DNTA_INSTRUMENT_ON
>>>> -DNTA_PYTHON_SUPPORT=2.7 -DNTAPY_INCLUDE="<python2.7/"
>>>> -DNTAPY_PACKAGES="\"lib/python2.7/site-packages\"" -I/usr/include
>>>> -I/lib/python2.7/site-packages/numpy/core/include -O1 -falign-functions
>>>> -DSWIG_TYPE_TABLE=_nupic_engine_internal -DNUPIC2 -MT
>>>> _engine_internal_la-engine_internal_py.lo -MD -MP -MF
>>>> .deps/_engine_internal_la-engine_internal_py.Tpo -c -o
>>>> _engine_internal_la-engine_internal_py.lo `test -f
>>>> 'engine_internal_py.cpp' || echo
>>>> '/home/pi/nupic/lang/py/bindings/engine_internal/'`engine_internal_py.cpp
>>>> libtool: compile: g++ -DHAVE_CONFIG_H -I.
>>>> -I/home/pi/nupic/lang/py/bindings/engine_internal
>>>> -I../../../../build_system/unix -I/usr/include/python2.7 -I/home/pi/nupic
>>>> -I/home/pi/nupic/external/linux32/include -DNUPIC2 -I/home/pi/nupic
>>>> -I/home/pi/nupic/external/common/include
>>>> -I/home/pi/nupic/external/linux32/include -fvisibility=hidden
>>>> -DNTA_INTERNAL -g -DNTA_ASSERTIONS_ON -fPIC -DPIC -ffloat-store -D_PY27
>>>> -DNTA_PLATFORM_linux32 -Wall -Wreturn-type -Wunused-variable
>>>> -Wno-deprecated -DBOOST_NO_WREGEX -DNTA_INSTRUMENT_ON
>>>> -DNTA_PYTHON_SUPPORT=2.7 "-DNTAPY_INCLUDE=<python2.7/"
>>>> -DNTAPY_PACKAGES=\"lib/python2.7/site-packages\" -I/usr/include
>>>> -I/lib/python2.7/site-packages/numpy/core/include -O1 -falign-functions
>>>> -DSWIG_TYPE_TABLE=_nupic_engine_internal -DNUPIC2 -MT
>>>> _engine_internal_la-engine_internal_py.lo -MD -MP -MF
>>>> .deps/_engine_internal_la-engine_internal_py.Tpo -c engine_internal_py.cpp
>>>> -fPIC -DPIC -o .libs/_engine_internal_la-engine_internal_py.o
>>>> engine_internal_py.cpp:4455:22: error: redefinition of 'struct
>>>> swig::traits<unsigned int>'
>>>> engine_internal_py.cpp:4378:22: error: previous definition of 'struct
>>>> swig::traits<unsigned int>'
>>>> engine_internal_py.cpp:4459:23: error: redefinition of 'struct
>>>> swig::traits_asval<unsigned int>'
>>>> engine_internal_py.cpp:4382:23: error: previous definition of 'struct
>>>> swig::traits_asval<unsigned int>'
>>>> engine_internal_py.cpp:4465:23: error: redefinition of 'struct
>>>> swig::traits_from<unsigned int>'
>>>> engine_internal_py.cpp:4388:23: error: previous definition of 'struct
>>>> swig::traits_from<unsigned int>'
>>>> make[2]: *** [_engine_internal_la-engine_internal_py.lo] Error 1
>>>>
>>>> I have not used swig before. Are there any swig experts out there that
>>>> can point me the right direction?
>>>>
>>>> Thanks,
>>>>
>>>> Matt
>>>>
>>>> On Aug 19, 2013, at 2:47 AM, Marek Otahal <[email protected]> wrote:
>>>>
>>>>> Hi Matt,
>>>>>
>>>>>
>>>>> On Sun, Aug 18, 2013 at 8:33 AM, Matt Keith <[email protected]> wrote:
>>>>> Thank you everyone for the feedback.
>>>>>
>>>>> Marek, yes I'll have to take a look at the linux32 support you have
>>>>> submitted. Won't there be issues with those .a files working on arm
>>>>> instead of x86? Should we create a linux32arm folder as well, or can we
>>>>> remove all dependancies on static libraries?
>>>>> I was wondering that too. Could you test, take the .a files from my
>>>>> linux32 branch and place them in your ARM environment?
>>>>> (I think it will be a prob and we'll need another platform to support)
>>>>>
>>>>> about removing all the static deps: I'm not sure now, as there are
>>>>> tickets like linux64/darwin64-environment, ... But before I was told we
>>>>> want to keep nupic as much self-contined as possible.(but building some
>>>>> of the libs might be tricky, that's why the precompiled archives .a are
>>>>> present) Maybe Scott can explain?
>>>>>
>>>>> PS: did you succeed removing the old Grouper classes, as Subutai
>>>>> mentioned here? I tried just rm the related files, but it was quite
>>>>> connected to other code too (so I wonder if that code is obsolete too,
>>>>> and can be removed, or need be reworked)
>>>>>
>>>>> Regards, Mark
>>>>>
>>>>>
>>>>> --
>>>>> Marek Otahal :o)
>>>>> _______________________________________________
>>>>> nupic mailing list
>>>>> [email protected]
>>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>>
>>>>
>>>> _______________________________________________
>>>> nupic mailing list
>>>> [email protected]
>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>>
>>>>
>>>> _______________________________________________
>>>> nupic mailing list
>>>> [email protected]
>>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>
>>>
>>> _______________________________________________
>>> nupic mailing list
>>> [email protected]
>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>>
>>>
>>> _______________________________________________
>>> nupic mailing list
>>> [email protected]
>>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>
>>
>> _______________________________________________
>> nupic mailing list
>> [email protected]
>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>>
>>
>> _______________________________________________
>> nupic mailing list
>> [email protected]
>> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
>
> _______________________________________________
> nupic mailing list
> [email protected]
> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
>
>
> _______________________________________________
> nupic mailing list
> [email protected]
> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
>
>
>
> --
> Marek Otahal :o)
> _______________________________________________
> nupic mailing list
> [email protected]
> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
