Re: Blinky on nRF52 DEV via Black Magic Probe: SIGSEGV in os_pkg_init

[Christopher Collins]

Hi Peter,

On Fri, Jun 16, 2017 at 04:10:52PM -0700, Peter Jones wrote:
> It's very likely that I'm doing something wrong so I'm reaching out for
> help.  All I want to do right now is to get Blinky working on an nRF52.
> 
> I followed the tutorial and built the boot loader and blinky.  I then
> created an image so the boot loader would be happy about the magic
> number.
> 
> I used GDB and the Black Magic Probe to flash the boot loader ELF file
> to address offset 0x.  Then I used the `restore' command to flash the
> blinky IMG file to offset 0x8000.  (I debugged the jlink shell scripts
> to figure this out.)
> 
> At this point I would expect that resetting the board would cause the
> LED to blink once a second.  Of course, that's not happening.
> 
> So I started GDB, used the `file' command to load symbols from the boot
> loader ELF file, and restarted the boot loader with `run'.  That's when
> I get:
> 
> 
>   Program received signal SIGSEGV, Segmentation fault.
>   os_pkg_init () at repos/apache-mynewt-core/kernel/os/src/os.c:223
>   
> 
> I stepped through the code and I end up inside `os_start' which appears
> to only have `assert(0)' as its body (because `#if
> MYNEWT_VAL(OS_SCHEDULING)').

Your procedure sounds correct to me, so I'm afraid I don't have an
explanation.  I am not familiar with the black magic probe, though.

The behavior you described is definitely unexpected.  The boot loader
should not call os_start() at all; it should jump to the image's start
up code instead.

Could you paste a stack trace at the point of failure?

Thanks,
Chris

Re: What are the best config settings to reduce ram usage, so that an app runs in 16 kb with least impact on functionality?

[Christopher Collins]

On Fri, Jun 16, 2017 at 10:08:21PM +0200, Alfred Schilken wrote:
> BTW:
> every now and then I get this timeout error:
> 
> newtmgr -c ble0c stat ble_gattc
> Error: Timeout waiting for host <-> controller sync
> 
> I’m using a microbit running the blehci, connected via adafruit FT232H with 
> rts/cts.

The BLE host (running in newtmgr) thinks the controller is unresponsive.
I have not seen this happend for a while.  Does the problem eventually
resolve itself?  Does resetting the controller make the problem go away?

Chris

Re: What are the best config settings to reduce ram usage, so that an app runs in 16 kb with least impact on functionality?

[Christopher Collins]

Hi Alfred,

On Fri, Jun 16, 2017 at 07:02:47PM +0200, Alfred Schilken wrote:
> Hello all, I’m experimenting with mynewt for several weeks now and I’m 
> especially impressed by the statistics, logging and image update 
> functionality.
> I’m having problems with getting all the statistics using newtmgr via BLE.
> 
> I  took bleprph as base, added ADC and enabled several stats, logs and so on.
> To reduce flash size I had to switch off debug and reduce the logging.
> 
> When I started the program I got an Assert-reboot-loop.
> The target board is a bbc microbit.
> 
> This seems to be cause of the error:
> ble_att_svr_entry_mem = malloc(
> OS_MEMPOOL_BYTES(ble_hs_max_attrs, sizeof (struct ble_att_svr_entry)));
> if (ble_att_svr_entry_mem == NULL) {
> rc = BLE_HS_ENOMEM;
> goto err;
> }

Fitting BLE and newtmgr into 16kB requires some creativity :).  There
are certainly some memory and mbuf optimizations that could be made, we
just need to go through the exercise of scouring the code.

> I tweaked these four config values to fix the crash-loop.
> BLE_ACL_BUF_COUNT: 4# was 4
> BLE_ACL_BUF_SIZE: 128  # was 255
> MSYS_1_BLOCK_COUNT: 4  # was 12
> MSYS_1_BLOCK_SIZE: 292 # was 292
> 
> The program boots now, I can see and connect to the BLE services and all this 
> is fine.
> 
> If I use newtmgr via ble transport to read the statistics, some of them are 
> responding, others return just Error: 2.
> mpstat and taskstat also don’t work.
> But „image list“ is working, I could even upload a new image.
> 
> 
> My question is:
> What are the best config settings to reduce ram usage, so that an app runs in 
> 16 kb with least impact on functionality? 

I was able to get mpstats working on a 16kB device with the following
mbuf settings:

MSYS_1_BLOCK_COUNT: 11
MSYS_1_BLOCK_SIZE: 200

> Some hints to reduce flash size would also be appreciated :-) 

You might want to look into the split image functionality:
https://mynewt.apache.org/latest/os/modules/split/split.  This allows
you dedicate more flash space to your application code.  I don't think
this will help with your immediates issues, however, since BLE and
newtmgr would both need to go into the loader image.

Chris

Re: Conditional compilation based on BSP name

[Christopher Collins]

On Wed, Jun 14, 2017 at 04:22:52PM +0200, Ugo Mastracchio wrote:
> Hello everyone, may I throw an absolute beginner's question ? 
> 
> How do I conditionally compile based on the BSP the target is
> associated with ?I want to use different GPIO pins depending on the board
> 
> Is there a system configuration setting valued with the BSP name ?

After writing my previous response, I am thinking I may have
misunderstood the question.  Generally, the PIN mappings are defined in
the BSP package itself, so there should be no need to remap pins based
on the BSP being used.  Are you perhaps trying to use the same BSP
package for two slightly different boards?

If this is what you want to do, you may want to take a look at how the
arduino_zero BSP handles this.  The 1) arduino zero and 2) arduino zero
pro hardware is very similar.  I believe the only difference are a few
GPIO mappings.  Rather than create a separate BSP for each board, the
arduino BSP package code uses conditional compilation.

Within the arduino repo
(https://github.com/runtimeco/mynewt_arduino_zero), the arduino_zero BSP
defines these settings:

BSP_ARDUINO_ZERO:
description: 'TBD'
value: 0
restrictions:
- "!BSP_ARDUINO_ZERO_PRO"

BSP_ARDUINO_ZERO_PRO:
description: 'TBD'
value: 0
restrictions:
- "!BSP_ARDUINO_ZERO"

Then, in hw/bsp/arduino_zero/include/bsp/bsp.h, pins are mapped as
follows:

#if MYNEWT_VAL(BSP_ARDUINO_ZERO_PRO)
 ARDUINO_ZERO_D2 = (8),
 ARDUINO_ZERO_D4 = (14),
#endif

#if MYNEWT_VAL(BSP_ARDUINO_ZERO)
 ARDUINO_ZERO_D2 = (14),
 ARDUINO_ZERO_D4 = (8),
#endif

It is up to the target package to define one (and only one) of
BSP_ARDUINO_ZERO_PRO or BSP_ARDUINO_ZERO.

This approach is nice because it eliminates the need for a lot of
duplicate code in a second BSP package.  One hassle involved is the
necessity to define the appropriate syscfg setting in the target
package.

Chris

Re: Cannot resolve dependency

[Christopher Collins]

On Fri, Jun 09, 2017 at 05:59:26PM -0700, Christopher Collins wrote:
> By the way, there is an unfortunate bug in the current version of newt.
> The dependency resolution error message gets the depender name wrong; it
> prints the dependee package instead!

I forgot to mention that this bug has since been fixed in the master
branch.

Chris

Re: Cannot resolve dependency

[Christopher Collins]

Hi Greg,

On Fri, Jun 09, 2017 at 05:47:11PM -0700, Greg Strange wrote:
> Hi,
> 
> I'm new to mynewt. I went through the quick start and have successfully
> installed mynewt, and have built and run the blinky project targeted to a
> NRF52 board.
> 
> As a next step I decided to copy the ble peripheral sample project into my
> own apps directory as a starting point for a simple BLE project. After
> doing so though the project will not build and I get this error:
> 
> *Building target targets/nrf52_blep*
> *Error: Could not resolve package dependency: @apache-mynewt-core/kernel/os
> ; depender: kernel/os*
> 
> 
> 
> A little more background  When I say "copy the ble peripheral sample
> project", what I did is copy apache-mynewt-core/apps/bleprph into apps, and
> renamed it "blep" because it's easier to type. I edited pkg.yml and added
> @apache-mynewt-core to all the dependencies so that pkg.yml now looks like
> this:
> 
> pkg.name: apps/blep
> pkg.type: app
> pkg.description: Simple BLE peripheral application.
> pkg.author: "Apache Mynewt "
> pkg.homepage: "http://mynewt.apache.org/;
> pkg.keywords:
> 
> pkg.deps:
> - "@apache-mynewt-core/boot/split"
> - "@apache-mynewt-core/kernel/os "

I believe the issue is caused by the trailing space above.

> - "@apache-mynewt-core/mgmt/imgmgr"
> - "@apache-mynewt-core/mgmt/newtmgr"
> - "@apache-mynewt-core/mgmt/newtmgr/transport/ble"
> - "@apache-mynewt-core/net/nimble/controller"
> - "@apache-mynewt-core/net/nimble/host"
> - "@apache-mynewt-core/net/nimble/host/services/ans"
> - "@apache-mynewt-core/net/nimble/host/services/gap"
> - "@apache-mynewt-core/net/nimble/host/services/gatt"
> - "@apache-mynewt-core/net/nimble/host/store/ram"
> - "@apache-mynewt-core/net/nimble/transport/ram"
> - "@apache-mynewt-core/sys/console/full"
> - "@apache-mynewt-core/sys/log/full"
> - "@apache-mynewt-core/sys/stats/full"
> - "@apache-mynewt-core/sys/sysinit"
> - "@apache-mynewt-core/sys/id"
> 
> 
> I also created a nrf52 target for blep.
> 
> Am I missing something?  If I point my target to the original bleprph
> project in the apache-mynewt-core directory, it builds fine.  But if I move
> it to my own apps directory I cannot get it to build.

That is a good way to get started with a new app.  I don't anticipate
any more errors after this one is solved.

By the way, there is an unfortunate bug in the current version of newt.
The dependency resolution error message gets the depender name wrong; it
prints the dependee package instead!

Chris

Re: Host Only Project

[Christopher Collins]

Hi Khaled,

On Wed, Jun 07, 2017 at 04:26:01PM +0200, Khaled Elsayed wrote:
[...]
> So, it has the controller and uart only. My questions are:
> 
> 1) What glues the different parts together? How does the code identify that
> when I include the transport/ram that host and link layer will communicate
> via RAM not uart. Where in the code is this handled?
> Similarly, if host, gap, gatt, etc are missing from pkg.yml where in the
> code are these modules avoided to be initialized or loaded (or more clearly
> where in the code are they loaded and initialized when they exist in the
> dependency)?

The answer to both questions lies in the sysinit() call.  The code that
ultimately gets executed by sysinit() is magically generated by newt
each time you build a target.  The sysinit function initializes each
package in the project.  Each package specifies zero or more "init"
functions, each with a corresponding stage number.  The generated
sysinit function is a flat sequence calls to each of these init
functions, ordered by stage.

The benefits of this mechanism are:
1. Apps can be configured and re-used without needing code
   modifications.
2. App code is simplified by the elimination of boilerplate
   initialization code.

The downside is, of course, generated code, which everyone hates :).

This is discussed in more detail at:
http://mynewt.apache.org/latest/os/modules/sysinitconfig/sysinitconfig/

This explains the initialization of the host, gap, and gatt packages.
If these packages get pulled in to your project, newt generates calls to
their initialization function in sysinit().

The BLE transport case is slightly more complicated.  Here, another
Mynewt concept is required: APIs.  

When a package declares that it supports a particular API, it promises
to the rest of the system that it exposes a set of functions and global
variables.  This allows another package to express an "abstract
dependency."  That is, a package depends on a particular interface, but
it doesn't care which package actually provides that interface.  The
collection of bluetooth HCI transports packages is an example of this.
The host and controller code is written such that they don't know the
specific transport being used; they just use the abstract API provided
by all the HCI transport packages.

The host package doesn't have a hard dependency on a specific transport
package.  Instead, it requires the "ble_transport" API.  At build time,
newt ensures that one (and only one) package provides the required API,
and it gives the host code access to the transport package's header
files.

Note: The newt tool doesn't verify that a package actually implements
the APIs is claims to.  The newt tool just makes sure that for each
required API, one and only one package that supplies the API gets pulled
in to the build.

At init time, the host package configures the transport package
(whichever one it may be) with function pointer callbacks.  In this way,
the host doesn't need to know which transport is actually being used,
and the transport doesn't need to depend on the host or controller.

> 2)  If one wants to use mynewt as host only with nordic soft device as is,
> certainly pkg.yml will not include net/nimble/controller. So, how to tell
> the build system to load or leave nordic controller in place?

You need to make sure your project does not depend on the controller
package.  Generally, there are two changes you would make to an app's
pkg.yml file to convert it from a combined-host-controller app to a
host-only app:

1. Remove from pkg.deps:
@apache-mynewt-core/net/nimble/controller
@apache-mynewt-core/net/nimble/transport/ram

2. Add your desired transport package to pkg.deps, e.g.,
@apache-mynewt-core/net/nimble/transport/uart

Chris

Re: #if directives and the MYNEWT_VAL(..) macro

[Christopher Collins]

Hi Sigve,

On Tue, Jun 06, 2017 at 01:01:20PM +0200, Sigve Sebastian Farstad wrote:
[...]
> So, there are perhaps two bugs in here. 1) why is MYNEWT_VAL_I2C_0 0 and
> not '0', as it is defined in syscfg.yml?

The single quotes are a YAML escaping mechanism similar to shell
quoting.  So, values of '0' and 0 are identical here.  The newt tool
just puts everything in single quotes to avoid potential issues with
special characters.

> And 2) I2C code is not included properly.
> 
> For 1), CfgEntry structs in the newt tool's source only support string
> entries, so I imagine something gets when going from yml entries to C
> literals.
> 
> One way of solving 2) is to use #ifdef instead of #if when checking if a
> value is defined. However, this doesn't work with the MYNEWT_VAL(..) macro,
> as far as I can tell, so one would need to use MYNEWT_VAL_I2C_0 directly
> instead.
> 
> Currently, I've worked around the issue by setting the I2C_0 value in
> syscfg.yml to something truthy (e.g. true), but it seems to me that the
> intention is for it to be e.g. 0 for I2C_0 and 1 for I2C_1, since the code
> in hal_i2c.c seems to want to use these numbers as array indexes as well.
> 
> Lastly, this probably affects more than just I2C as the #if MYNEWT_VAL(..)
> pattern is used many places in the core.
> 
> 
> Thoughts?

As others have indicated, this particular setting is meant to be a
boolean.  A value of 0 means the I2C port is unused.

I do think you are right in the general case, though: there is a
potential ambiguity.  It is not possible to test if a particular syscfg
setting is undefined:

1. If the package which defines a setting is not part of your project,
the corresponding MYNEWT_VAL_<...> macro won't be defined.

2. If an undefined macro is used in an `#if` context, it evaluates to 0.

So, the use of `#if` depends on the assumption that 0 and undefined are
equivalent in your code.

If you ever need to know if a MYNEWT_VAL setting is truly undefined, I
think you will need to bypass the MYNEWT_VAL() macro and test the value
directly with `#ifdef`, e.g.,

#ifdef MYNEWT_VAL_I2C_0

I don't think I have run into any cases where I've needed to distinguish
undefined from 0, though.

Chris

Re: segger jlink

[Christopher Collins]

Hi Pierre,

First, don't worry about asking a lot of questions.  There are probably
others with the same questions who will benefit from your asking them!

On Tue, May 30, 2017 at 04:32:20PM +0100, Pierre Kircher wrote:
[...]
> the breakpoint points on main.c 129 > apps/bleprph/src
> 
> Reading symbols from 
> /Users/pkircher/dev/puck/bin/targets/bleprph/app/apps/bleprph/bleprph.elf...
> done.
> os_tick_idle (ticks=595) at 
> repos/apache-mynewt-core/hw/mcu/nordic/nrf52xxx/src/hal_os_tick.c:200
> 200   if (ticks > 0) {
> Not implemented stop reason (assuming exception): undefined
> 
> never hits that breakpoint - is that a known issue ?

That is not a known issue.  My guess is that the API call immediately
before that point (ble_gap_adv_set_fields) is failing for some reason.
One common reason is that the app is trying to put too much data into
the advertisement.  For example, this happens if you are using a very
long name.  I would backtrack a bit; set a breakpoint on
bleprph_advertise() and see if your app gets there.

FYI, here is the expected sequence of events:
1. main()
2. bleprph_on_sync() (communication established among host and
   controller).
3. bleprph_advertise (called from bleprph_on_sync()).

Chris

Re: [VOTE] Graduation of podling Apache Mynewt to Top Level Project

[Christopher Collins]

+1

Chris

On Thu, May 25, 2017 at 05:55:26PM -0700, aditi hilbert wrote:
> Hi all,
> 
> We feel confident that we pass all the requirements to graduate.  Below is 
> the proposed Graduation Resolution (based on the standard template). A 
> separate thread of DISCUSS is also open for this.
> 
> The full text of the proposed resolution is as follows:
> 
> 
> The incubating Apache Mynewt community believes it is time to graduate to TLP.
> 
> Apache Mynewt entered incubation in October of 2015.  Since then, we've 
> overcome technical challenges and built Apache's first Operating System for 
> IoT devices, and made 6 releases.  Our most recent releases include NimBLE, 
> OIC1.1 support, MCUbootloader. We are a very helpful and engaged community, 
> ready to answer all questions and feedback directed to us via the user list.  
> We've added several committers from different organizations, and are actively 
> pursuing others. While we continue working on maturity, all projects are 
> ongoing processes, and we believe we no longer need the incubator to continue.
> 
> To inform the discussion, here is some basic project information:
> 
> Project status:
>   http://incubator.apache.org/projects/Mynewt.html
> 
> Project website:
>   http://Mynewt.incubator.apache.org/
> 
> Project documentation:
>  https://cwiki.apache.org/confluence/display/MYNEWT/Apache+Mynewt+Project
> 
> Maturity assessment:
> https://cwiki.apache.org/confluence/display/MYNEWT/Maturity+model
> 
> DRAFT of the board resolution is at the bottom of this email
> 
> Proposed PMC size: 20 members
> 
> Total number of committers: 20 members
> 
> Various PMC/Committer affiliations (* indicated chair)
>Runtime (6)
>Google (1)
>Imagination Technologies (1)
>Codecoup (4)
>Adafruit (1)
> 
> 4,565 commits on develop
> 280 closed PR”s on GitHub
> 39 contributors across all branches
> 92 forks on Github
> 
> dev list averaged ~200 msgs/month in the last one year
> 
> 647 issues created
> 465 issues resolved/closed
>  
> --
> Resolution:
> 
> Establish the Apache Mynewt Project
> 
> WHEREAS, the Board of Directors deems it to be in the best
> interests of the Foundation and consistent with the
> Foundation's purpose to establish a Project Management
> Committee charged with the creation and maintenance of
> open-source software, for distribution at no charge to the
> public, related to a data management platform that provides
> real-time, consistent access to data-intensive applications
> throughout widely distributed cloud architectures.
> 
> NOW, THEREFORE, BE IT RESOLVED, that a Project Management
> Committee (PMC), to be known as the "Apache Mynewt Project",
> be and hereby is established pursuant to Bylaws of the
> Foundation; and be it further
> 
> RESOLVED, that the Apache Mynewt Project be and hereby is
> responsible for the creation and maintenance of software
> related to an embedded OS optimized for networking and built for remote 
> management of constrained devices that are incapable of running either Linux 
> or Android. 
> 
> RESOLVED, that the office of "Vice President, Apache Mynewt" be
> and hereby is created, the person holding such office to
> serve at the direction of the Board of Directors as the chair
> of the Apache Mynewt Project, and to have primary responsibility
> for management of the projects within the scope of
> responsibility of the Apache Mynewt Project; and be it further
> 
> RESOLVED, that the persons listed immediately below be and
> hereby are appointed to serve as the initial members of the
> Apache Mynewt Project: 
> *Justin Mclean” <jmcl...@apache.org> 
> *P. Taylor Goetz” <ptgo...@apache.org>, 
> *Greg Stein <gst...@apache.org>, 
> *Jim Jagielski <j...@apache.org>, 
> *Sterling Hughes <sterl...@apache.org>, 
> *Marko Kiiskila <ma...@apache.org>, 
> *will sanfilippo <w...@apache.org>, 
> Christopher Collins <ccoll...@apache.org>, 
> Vipul Rahane <vipulrah...@apache.org>, 
> Fabio Utzig <ut...@apache.org>, 
> Andrzej Kaczmarek <a...@apache.org>, 
> Michał Narajowski <na...@apache.org>, 
> Szymon Janc <j...@apache.org>, 
> Łukasz Rymanowski <ry...@apache.org>, 
> Neel Natu <n...@apache.org>, 
> Peter Snyder <pete...@apache.org>, 
> Paul Dietrich <paulfdietr...@apache.org, 
> Julian Ingram <jul...@apache.org>, 
> Kevin Townsend <kt...@apache.org>,
> Aditi Hilbert <ad...@apache.org>
> 
> NOW, THEREFORE, BE IT FURTHER RESOLVED, that Justin Mcclean
> be appointed to the office of Vice President, Apache M

Re: Problems with erasing flash on nordic devices

[Christopher Collins]

On Tue, May 16, 2017 at 05:11:36PM -0700, Jacob Rosenthal wrote:
> Seems like it could go in flash_map?
> 
> Its also been requested to also have a function that detects if an area
> should be erased, based on it having any non 0xff bytes within.
> 
> Should that go there as well.

That is the same thing, isn't it?

> Whats a good method to loop through all bytes of a flash area?

Perhaps something like the following:

1. Allocate a buffer that is properly aligned to be accessed by an
unsigned int*.  It may take some testing to
determine a good size that balances speed and size efficiency; I would
start with 64 bytes.  I would also start by putting this buffer on the
stack.

2. Read a chunk of flash into the buffer
(min(size-of-buffer, num-unread-bytes).

3. Use an unsigned int* to iterate through the buffer.  If the pointer
points to something other than (unsigned int)-1, then the flash is not
erased.

4. If there are any remaining bytes (i.e., number of bytes read is not a
multiple of sizeof(unsigned int)), compare them individually against
0xff using a uint8_t*.

Chris

Re: Problems with erasing flash on nordic devices

[Christopher Collins]

Hi Jacob,

On Mon, May 15, 2017 at 02:51:35PM -0700, Jacob Rosenthal wrote:
> So Ill duplicate the upload command to an erase command.
> 
> Is there a bit of metadata I could check to see if a slot was filled and
> erase was necessary? If so I could conditionally erase in the upload
> command still, for backwards compat reasons.

No, I'm afraid there is no way to distinguish an empty slot from a
partially written one.  We would need to add a new newtmgr command for
that (or add an option to "image list").

Just to be clear, there also isn't an erase command at the moment.  This
still needs to be added.

Chris

Re: Reducing GATT write attribute's timeout and read attribute's BLE_HS_ENOMEM

[Christopher Collins]

On Mon, May 15, 2017 at 11:01:38AM -0700, Christopher Collins wrote:
> Hi Łukasz,
> 
> On Mon, May 15, 2017 at 12:33:59PM +0100, Łukasz Wolnik wrote:
> > Hello,
> > 
> > From time to time my ble_gattc_write_flat (run as central) is timing out
> > after 20 seconds while sending to an Android 6 phone (in peripherial mode).
> > Is there a way to reduce the timeout to just 1 second? At the moment if
> > there's an issue with writing, my newt program has to wait 20 seconds until
> > it can respond to a timeout.
> > 
> > What's the best strategy here? Keep "bombarding" the peripherial with
> > multiple writes until receiving first confirmation. Or reduce the timeout
> > from 20 seconds (I don't know where this value is coming from) and resend
> > only when got an HCI 19 timeout error in the callback?

Oops, I forgot to respond to your actual question :).  Sorry about that.
The 30-second timeout is hardcoded in the spec, and is currently not
configurable (Vol. 3, Part F, 3.3.3).  It might be useful to make this
configurable, but the device would not be standards compliant.

Chris

Re: Reducing GATT write attribute's timeout and read attribute's BLE_HS_ENOMEM

[Christopher Collins]

Hi Łukasz,

On Mon, May 15, 2017 at 12:33:59PM +0100, Łukasz Wolnik wrote:
> Hello,
> 
> From time to time my ble_gattc_write_flat (run as central) is timing out
> after 20 seconds while sending to an Android 6 phone (in peripherial mode).
> Is there a way to reduce the timeout to just 1 second? At the moment if
> there's an issue with writing, my newt program has to wait 20 seconds until
> it can respond to a timeout.
> 
> What's the best strategy here? Keep "bombarding" the peripherial with
> multiple writes until receiving first confirmation. Or reduce the timeout
> from 20 seconds (I don't know where this value is coming from) and resend
> only when got an HCI 19 timeout error in the callback?

The lower layers should ensure all packets are delivered, so the
application should not perform any retries.  In fact, once an ATT
exchange times out, the connection is terminated.

I think the problem is the resource exhaustion, as indicated by the
BLE_HS_ENOMEM codes you are seeing.  This particular error code is,
unfortunately, quite general; it applies to several different resource
types.  Since your device is initiating several GATT procedures in
parallel, my guess is that the problematic resource is one of the
following:

* mbufs
* GATT client procedure objects

If you have gdb attached to your device, you can check if either of
these is the problem.  Let me know if you are interested in this.

I would recommend just increasing the following setting and seeing what
happens:

MSYS_1_BLOCK_COUNT  # default: 12
BLE_GATT_MAX_PROCS  # default: 4

As an aside, this issue tells me two things:
1. We need to make sure there are statistics available that indicate
   failure to allocate each resource type.
2. We should log the exact cause of failure when a procedure fails.

Chris

> My logs are below:
> 
> * CONNECTED 3 *
> 007034 [ts=54953064ssb, mod=64 level=1] peer's connection handle;
> conn_handle=3; addr=2000719b; attr_read=0 attr_write=0
> 007037 [ts=54976500ssb, mod=4 level=1] GATT procedure initiated: write;
> att_handle=42 len=4
> 007039 [ts=54992124ssb, mod=4 level=1] GAP procedure initiated: discovery;
> own_addr_type=0 filter_policy=0 passive=1 limited=0 filter_duplicates=0
> duration=forever
> 007059 [ts=55148428ssb, mod=64 level=1] wb_on_write msg=5, err=0
> 007060 [ts=55156240ssb, mod=64 level=1] Error: Failed to read
> characteristic; rc=6
> 
> 
> Again sometimes I'm getting BLE_HS_ENOMEM error while trying to read
> attribute from a different handle than the write one.
> 
> My syscfg.yml:
> 
> LOG_LEVEL: 1
> UART_0: 1
> 
> # Default task settings
> OS_MAIN_STACK_SIZE: 512
> 
> # BLE settings
> BLE_MAX_CONNECTIONS: 4
> 
> # Make BLE more stable
> BLE_LL_CFG_FEAT_DATA_LEN_EXT: 0
> 
> BLE_GATT_READ_MAX_ATTRS: 2
> BLE_GATT_WRITE_MAX_ATTRS: 2
> 
> I have tried reducing OS_MAIN_STACK_SIZE to 384, also tried with
> BLE_MAX_CONNECTIONS to 3, etc. This read BLE_HS_ENOMEM seems to pop up in a
> random manner.
> 
> Is there a way to check how much of device's RAM has been already used
> (nrf52xx)?
> 
> Kind regards,
> Łukasz

Re: Subscribing to peer notifications

[Christopher Collins]

Hi Simon,

On Wed, May 10, 2017 at 12:47:43PM -0700, Simon Ratner wrote:
> Hi dev list,
> 
> I may be completely missing this, but is there a gattc API for subscribing
> to notifications on a NOTIFY characteristic exposed by a peer peripheral?
> 
> Cheers,
> simon

You subscribe by writing a special value to the characteristic's CCCD:
* notifications:0x01:0x00
* indications:  0x02:0x00

You can do this via ble_gattc_write() (or more conveniently,
ble_gattc_write_flat()).  For an example of how this is done, see the
blecent_read_write_subscribe() function in apps/blecent/src/main.c.

The host API could probably benefit from a few convenience functions
here.

Chris

Re: Problems with erasing flash on nordic devices

[Christopher Collins]

On Fri, May 05, 2017 at 05:58:52PM -0700, Jacob Rosenthal wrote:
> Thats basically what Im saying. Though  if you just do #2 I wont need a
> separate erase command. It'll erase and disconnect the first time, but not
> the second time. Ill get an erase command for free.

I don't think that will work without some more intelligence in the image
upload command handler.  Here is the typical sequence of events:

1. Image upload (offset=0) command received.
2. Command handler erases second image slot.
3. BLE connection drops during erase.
4. Command handler writes first part of image to beginning of second
   slot.

If you try to reconnect and upload again, the firmware will need to
erase the second slot once again, as the first image part has already
been written.  By separating the flash erase from the write, you can
ensure the slot is empty before starting the upload.

The extra "intelligence" I mentioned would be the command handler
comparing the flash contents against the data contained in the upload
request.  If the contents are identical, the firmware could skip the
erase and act like it just wrote the image segment (or indicate to the
client that it should resume the upgrade at a particular offset).

Chris

Re: Question about BLE GATT Services

[Christopher Collins]

On Wed, May 03, 2017 at 12:10:28PM -0700, Christopher Collins wrote:
> On Wed, May 03, 2017 at 10:59:59AM -0700, Pritish Gandhi wrote:
> > Hi All,
> > Just wanted to give an update to the community on the status of trying to
> > start the GATT server after the OS is initialized.
> > 
> > to ble_hs.c
> > @@ -606,5 +603,7 @@ ble_hs_init(void)
> >  /* Configure the HCI transport to communicate with a host. */
> >  ble_hci_trans_cfg_hs(ble_hs_hci_rx_evt, NULL, ble_hs_rx_data, NULL);
> > 
> > +#if 0 /* Let the Application set the queue and start the BLE stack */
> >  ble_hs_evq_set(os_eventq_dflt_get());
> > +#endif
> >  }
> 
> Is there a particular reason you removed the call to ble_hs_evq_set(),
> rather than the call to ble_hs_start()?  I think it is preferable to
> keep ble_hs_evq_set() here, and just hold off on starting the host.  If
> some other package tries to use the host before its eventq is set, it
> will result in a null pointer dereference.
> 
> > Then, I just have to call ble_hs_evq_set() with my ble thread's queue after
> > registering my GATT services.
> > 
> > The next thing I need to figure out though, is how to remove registered
> > GATT services and add new ones after the GATT server has started.
> > I think ble_hs_sched_reset() might work, if I move its function definition
> > from src/ble_hs_priv.h to include/ble_hs.h.
> > I haven't tried this out yet though. Will get to it soon.
> 
> Unfortunately, I don't think ble_hs_sched_reset() will work here.  That
> function just resets the controller and connection / procedure lists,
> but it does not have any effect on the registered GATT services.]
> 
> I haven't tried it, but I think the following sequence should do the
> trick:
> 
> 1. Register your new services with ble_gatts_add_svcs().
> 2. Call ble_gatts_start()
> 
> ble_gatts_start() should already get called at startup as soon as the
> host starts.  When this happens, the registered services get exposed to
> BLE peers, and the "working set" of services is cleared.  This is why
> you can just start adding new services in step 1; the working set has
> been cleared, so you are starting from scratch.  When you call
> ble_gatts_start() in step 2, the previously-exported services get
> replaced with the new ones.

Actually, there is one more thing that you'll need to do here: set
ble_gatts_num_cfgable_chrs to 0.  This is a static variable, so you'll
need to hack the code a bit to get access to it.  So the sequence should be:

1. ble_gatts_num_cfgable_chrs = 0;
2. Register your new services with ble_gatts_add_svcs().
3. Call ble_gatts_start()

Without this step, the GATT server will try to allocate too many CCCD
entries, which could cause the ble_gatts_start() call to fail.

Chris

Re: Question about BLE GATT Services

[Christopher Collins]

On Wed, May 03, 2017 at 10:59:59AM -0700, Pritish Gandhi wrote:
> Hi All,
> Just wanted to give an update to the community on the status of trying to
> start the GATT server after the OS is initialized.
> 
> to ble_hs.c
> @@ -606,5 +603,7 @@ ble_hs_init(void)
>  /* Configure the HCI transport to communicate with a host. */
>  ble_hci_trans_cfg_hs(ble_hs_hci_rx_evt, NULL, ble_hs_rx_data, NULL);
> 
> +#if 0 /* Let the Application set the queue and start the BLE stack */
>  ble_hs_evq_set(os_eventq_dflt_get());
> +#endif
>  }

Is there a particular reason you removed the call to ble_hs_evq_set(),
rather than the call to ble_hs_start()?  I think it is preferable to
keep ble_hs_evq_set() here, and just hold off on starting the host.  If
some other package tries to use the host before its eventq is set, it
will result in a null pointer dereference.

> Then, I just have to call ble_hs_evq_set() with my ble thread's queue after
> registering my GATT services.
> 
> The next thing I need to figure out though, is how to remove registered
> GATT services and add new ones after the GATT server has started.
> I think ble_hs_sched_reset() might work, if I move its function definition
> from src/ble_hs_priv.h to include/ble_hs.h.
> I haven't tried this out yet though. Will get to it soon.

Unfortunately, I don't think ble_hs_sched_reset() will work here.  That
function just resets the controller and connection / procedure lists,
but it does not have any effect on the registered GATT services.]

I haven't tried it, but I think the following sequence should do the
trick:

1. Register your new services with ble_gatts_add_svcs().
2. Call ble_gatts_start()

ble_gatts_start() should already get called at startup as soon as the
host starts.  When this happens, the registered services get exposed to
BLE peers, and the "working set" of services is cleared.  This is why
you can just start adding new services in step 1; the working set has
been cleared, so you are starting from scratch.  When you call
ble_gatts_start() in step 2, the previously-exported services get
replaced with the new ones.

Chris

Re: Config & FS command not working on newtmgr

[Christopher Collins]

On Sat, Apr 29, 2017 at 01:41:55AM +, Lm Chew wrote:
> Hi Chris,
> 
> 
> Ops... i did missed the FS_NMGR config, thanks.
> 
> 
> Now the FS command works for download.
> 
> 
> When i try to upload the same file back it just got stuck at "Reading from 
> data channel" as shown in the log below.
> 
> 
> newtmgr -c myserial fs upload test.txt /cfg/test.txt -ldebug
> 
> >>>
> 2017/04/29 09:31:51 [DEBUG] Writing newtmgr request &{Op:2 Flags:0 Len:7 
> Group:0 Seq:0 Id:1 Data:[161 100 101 99 104 111 0]}
> 2017/04/29 09:31:51 [DEBUG] Serializing request &{Op:2 Flags:0 Len:7 Group:0 
> Seq:0 Id:1 Data:[161 100 101 99 104 111 0]} into buffer [2 0 0 7 0 0 0 1 161 
> 100 101 99 104 111 0]
> 2017/04/29 09:31:51 [DEBUG] Tx packet dump:
>   02 00 00 07 00 00 00 01  a1 64 65 63 68 6f 00 |.decho.|
> 
> 2017/04/29 09:31:51 [DEBUG] Writing [6 9] to data channel
> 2017/04/29 09:31:51 [DEBUG] Writing [65 66 69 67 65 65 65 72 65 65 65 65 65 
> 97 70 107 90 87 78 111 98 119 68 51 122 81 61 61] to data channel
> 2017/04/29 09:31:51 [DEBUG] Writing [10] to data channel
> 2017/04/29 09:31:51 [DEBUG] Reading [] from data channel
> 2017/04/29 09:31:51 [DEBUG] Reading [6 9 65 65 119 68 65 65 65 67 65 65 65 65 
> 65 98 47 47 57 116 56 61] from data channel
> 2017/04/29 09:31:51 [DEBUG] Rx packet dump:
>   03 00 00 02 00 00 00 01  bf ff|..|
> 
> 2017/04/29 09:31:51 [DEBUG] Deserialized response &{Op:3 Flags:0 Len:2 
> Group:0 Seq:0 Id:1 Data:[191 255]}
> 2017/04/29 09:31:51 [DEBUG] Writing newtmgr request &{Op:2 Flags:0 Len:39 
> Group:8 Seq:0 Id:0 Data:[164 100 100 97 116 97 67 49 50 51 99 108 101 110 22 
> 100 110 97 109 101 109 47 99 102 103 47 116 101 115 116 46 116 120 116 99 111 
> 102 102 0]}
> 2017/04/29 09:31:51 [DEBUG] Serializing request &{Op:2 Flags:0 Len:39 Group:8 
> Seq:0 Id:0 Data:[164 100 100 97 116 97 67 49 50 51 99 108 101 110 22 100 110 
> 97 109 101 109 47 99 102 103 47 116 101 115 116 46 116 120 116 99 111 102 102 
> 0]} into buffer [2 0 0 39 0 8 0 0 164 100 100 97 116 97 67 49 50 51 99 108 
> 101 110 22 100 110 97 109 101 109 47 99 102 103 47 116 101 115 116 46 116 120 
> 116 99 111 102 102 0]
> 2017/04/29 09:31:51 [DEBUG] Tx packet dump:
>   02 00 00 27 00 08 00 00  a4 64 64 61 74 61 43 31  |...'.ddataC1|
> 0010  32 33 63 6c 65 6e 16 64  6e 61 6d 65 6d 2f 63 66  |23clen.dnamem/cf|
> 0020  67 2f 74 65 73 74 2e 74  78 74 63 6f 66 66 00 |g/test.txtcoff.|
> 
> 2017/04/29 09:31:51 [DEBUG] Writing [6 9] to data channel
> 2017/04/29 09:31:51 [DEBUG] Writing [65 68 69 67 65 65 65 110 65 65 103 65 65 
> 75 82 107 90 71 70 48 89 85 77 120 77 106 78 106 98 71 86 117 70 109 82 117 
> 89 87 49 108 98 83 57 106 90 109 99 118 100 71 86 122 100 67 53 48 101 72 82 
> 106 98 50 90 109 65 65 78 83] to data channel
> 2017/04/29 09:31:51 [DEBUG] Writing [10] to data channel
> 2017/04/29 09:31:51 [DEBUG] Reading [48 48 56 48 57 50 32 85 110 104 97 110 
> 100 108 101 100 32 105 110 116 101 114 114 117 112 116 32 40 51 41 44 32 101 
> 120 99 101 112 116 105 111 110 32 115 112 32 48 120 50 48 48 48 48 56 100 56] 
> from data channel
> 2017/04/29 09:31:51 [DEBUG] Reading [48 48 56 48 57 50 32 32 114 48 58 48 120 
> 50 48 48 48 48 101 56 52 32 32 114 49 58 48 120 48 48 48 48 48 48 48 48 32 32 
> 114 50 58 48 120 48 48 48 48 48 48 49 54 32 32 114 51 58 48 120 48 48 48 48 
> 48 56 48 48] from data channel
> 2017/04/29 09:31:51 [DEBUG] Reading [48 48 56 48 57 50 32 32 114 52 58 48 120 
> 50 48 48 48 48 101 56 52 32 32 114 53 58 48 120 50 48 48 48 48 57 102 56 32 
> 32 114 54 58 48 120 50 48 48 48 50 57 97 52 32 32 114 55 58 48 120 48 48 48 
> 48 48 52 48 48] from data channel
> 2017/04/29 09:31:51 [DEBUG] Reading [48 48 56 48 57 50 32 32 114 56 58 48 120 
> 48 48 48 50 53 51 49 100 32 32 114 57 58 48 120 50 48 48 48 48 102 99 48 32 
> 114 49 48 58 48 120 48 48 48 50 57 54 101 48 32 114 49 49 58 48 120 48 48 48 
> 48 48 48 50 102] from data channel
> 2017/04/29 09:31:51 [DEBUG] Reading [48 48 56 48 57 50 32 114 49 50 58 48 120 
> 48 48 48 48 48 48 48 48 32 32 108 114 58 48 120 48 48 48 48 56 56 57 98 32 32 
> 112 99 58 48 120 48 48 48 48 48 56 48 48 32 112 115 114 58 48 120 52 48 48 48 
> 48 48 48 48] from data channel
> 2017/04/29 09:31:51 [DEBUG] Reading [48 48 56 48 57 50 32 73 67 83 82 58 48 
> 120 48 48 52 50 49 56 48 51 32 72 70 83 82 58 48 120 52 48 48 48 48 48 48 48 
> 32 67 70 83 82 58 48 120 48 48 48 50 48 48 48 48] from data channel
> 2017/04/29 09:31:51 [DEBUG] Reading [48 48 56 48 57 50 32 66 70 65 82 58 48 
> 120 101 48 48 48 101 100 51 56 32 77 77 70 65 82 58 48 120 101 48 48 48 101 
> 100 51 52] from data channel
> >>>

It looks like newtmgr is sending a lot of little fragments.  Your device
might be running out of mbufs before it can reassemble the newtmgr
request.  I would try increasing the MSYS_1_BLOCK_COUNT setting if you
have the RAM for it.

> The Config 

Re: Config & FS command not working on newtmgr

[Christopher Collins]

Hi Lm,

The device is reporting a newtmgr error of 5, which corresponds to
MGMT_ERR_ENOENT (Yes, we need better error reporting!).  This means the
device doesn't recognize the newtmgr command you sent it.

Did you enable file system newtmgr support?  The syscfg setting
"FS_NMGR" has to be set to 1.  You can see the value of this setting
with the "newt target config show " command.

Chris

On Fri, Apr 28, 2017 at 05:00:27PM +, Lm Chew wrote:
> Hi,
> 
> 
> 1. When i try to download a file i created on the device, i get the following 
> error when newtmgr crashes:
> 
> 
> newtmgr -c myserial fs download /cfg/test.txt test.txt -ldebug
> 
> 
> 2017/04/29 00:32:58 [DEBUG] Writing newtmgr request &{Op:0 Flags:0 Len:25 
> Group:8 Seq:0 Id:0 Data:[162 100 110 97 109 101 109 47 99 102 103 47 116 101 
> 115 116 46 116 120 116 99 111 102 102 0]}
> 2017/04/29 00:32:58 [DEBUG] Serializing request &{Op:0 Flags:0 Len:25 Group:8 
> Seq:0 Id:0 Data:[162 100 110 97 109 101 109 47 99 102 103 47 116 101 115 116 
> 46 116 120 116 99 111 102 102 0]} into buffer [0 0 0 25 0 8 0 0 162 100 110 
> 97 109 101 109 47 99 102 103 47 116 101 115 116 46 116 120 116 99 111 102 102 
> 0]
> 2017/04/29 00:32:58 [DEBUG] Tx packet dump:
>   00 00 00 19 00 08 00 00  a2 64 6e 61 6d 65 6d 2f  |.dnamem/|
> 0010  63 66 67 2f 74 65 73 74  2e 74 78 74 63 6f 66 66  |cfg/test.txtcoff|
> 0020  00|.|
> 
> 2017/04/29 00:32:58 [DEBUG] Writing [6 9] to data channel
> 2017/04/29 00:32:58 [DEBUG] Writing [65 67 77 65 65 65 65 90 65 65 103 65 65 
> 75 74 107 98 109 70 116 90 87 48 118 89 50 90 110 76 51 82 108 99 51 81 117 
> 100 72 104 48 89 50 57 109 90 103 66 78 111 65 61 61] to data channel
> 2017/04/29 00:32:58 [DEBUG] Writing [10] to data channel
> 2017/04/29 00:32:58 [DEBUG] Reading [] from data channel
> 2017/04/29 00:32:58 [DEBUG] Reading [6 9 65 66 65 66 65 65 65 71 65 65 103 65 
> 65 76 57 105 99 109 77 70 47 48 121 53] from data channel
> 2017/04/29 00:32:58 [DEBUG] Rx packet dump:
>   01 00 00 06 00 08 00 00  bf 62 72 63 05 ff|.brc..|
> 
> 2017/04/29 00:32:58 [DEBUG] Deserialized response &{Op:1 Flags:0 Len:6 
> Group:8 Seq:0 Id:0 Data:[191 98 114 99 5 255]}
> 2017/04/29 00:32:58 [DEBUG] goroutine 1 [running]:
> mynewt.apache.org/newt/newtmgr/vendor/mynewt.apache.org/newt/util.NewNewtError(0xc820136ce0,
>  0xf, 0xc820133a60)
> /home/dev/dev/go/src/mynewt.apache.org/newt/newtmgr/vendor/mynewt.apache.org/newt/util/util.go:75
>  +0x103
> mynewt.apache.org/newt/newtmgr/protocol.DecodeFileDownloadResponse(0xc820136c70,
>  0x6, 0x8, 0x0, 0x0, 0x0)
> /home/dev/dev/go/src/mynewt.apache.org/newt/newtmgr/protocol/fsdownload.go:90 
> +0x424
> mynewt.apache.org/newt/newtmgr/cli.fsDownloadCmd(0xc8201258c0, 0xc82001ac80, 
> 0x2, 0x5)
> /home/dev/dev/go/src/mynewt.apache.org/newt/newtmgr/cli/fs.go:161 +0x715
> mynewt.apache.org/newt/newtmgr/vendor/github.com/spf13/cobra.(*Command).execute(0xc8201258c0,
>  0xc82001ac30, 0x5, 0x5, 0x0, 0x0)
> /home/dev/dev/go/src/mynewt.apache.org/newt/newtmgr/vendor/github.com/spf13/cobra/command.go:636
>  +0x8e9
> mynewt.apache.org/newt/newtmgr/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc820124240,
>  0xc8201258c0, 0x0, 0x0)
> /home/dev/dev/go/src/mynewt.apache.org/newt/newtmgr/vendor/github.com/spf13/cobra/command.go:722
>  +0x55c
> mynewt.apache.org/newt/newtmgr/vendor/github.com/spf13/cobra.(*Command).Execute(0xc820124240,
>  0x0, 0x0)
> /home/dev/dev/go/src/mynewt.apache.org/newt/newtmgr/vendor/github.com/spf13/cobra/command.go:681
>  +0x2d
> main.main()
> /home/dev/dev/go/src/mynewt.apache.org/newt/newtmgr/newtmgr.go:25 +0x25
> 
> goroutine 17 [syscall, locked to thread]:
> runtime.goexit()
> /usr/lib/go-1.6/src/runtime/asm_amd64.s:1998 +0x1
> 
> goroutine 5 [syscall]:
> os/signal.signal_recv(0x0)
> /usr/lib/go-1.6/src/runtime/sigqueue.go:116 +0x132
> os/signal.loop()
> /usr/lib/go-1.6/src/os/signal/signal_unix.go:22 +0x18
> created by os/signal.init.1
> /usr/lib/go-1.6/src/os/signal/signal_unix.go:28 +0x37
> 
> Error: Target error: 5
> 
> 
> 2. When i try to write a config to /cfg/run i get the following response:
> 
> newtmgr -c myserial config testing "123" -ldebug
> 
> 
> 
> 
> 2017/04/29 00:33:36 [DEBUG] Writing newtmgr request &{Op:2 Flags:0 Len:22 
> Group:3 Seq:0 Id:0 Data:[162 100 110 97 109 101 103 116 101 115 116 105 110 
> 103 99 118 97 108 99 49 50 51]}
> 2017/04/29 00:33:36 [DEBUG] Serializing request &{Op:2 Flags:0 Len:22 Group:3 
> Seq:0 Id:0 Data:[162 100 110 97 109 101 103 116 101 115 116 105 110 103 99 
> 118 97 108 99 49 50 51]} into buffer [2 0 0 22 0 3 0 0 162 100 110 97 109 101 
> 103 116 101 115 116 105 110 103 99 118 97 108 99 49 50 51]
> 2017/04/29 00:33:36 [DEBUG] Tx packet dump:
>   02 00 00 16 00 03 00 00  a2 64 6e 61 6d 65 67 74  |.dnamegt|
> 0010  65 73 74 69 6e 67 63 76  61 6c 63 31 

Re: newtmgr image upload nrf51dk disconnects with reason=8

[Christopher Collins]

On Fri, Apr 21, 2017 at 09:29:59PM -0700, Simon Ratner wrote:
> > itvl_min and itvl_max specify the range of connection interval values that
> > you're willing to accept. The controller chooses the value in that range
> > that it likes best. If you want an exact interval, then specify the same
> > number for both fields.
> 
> Note that iOS will reject such a request, see their accessory design
> guidelines.
> For one, itvl_max must be at least 20ms bigger than itvl_min.

Thanks, that is good to know.  I should also take this opportunity to
correct something I wrote above.  I said it is up to the slave
controller to select its preferred interval from the range specified by
the host.  This is wrong.  The slave sends both the min and max interval
values to the master, and the master chooses the interval to use (or
rejects the request entirely).  So this is how iOS can impose the
restriction you mentioned even though the iphone is acting as the master
(central).  Sorry for the misinformation.

Chris

Re: newtmgr image upload nrf51dk disconnects with reason=8

[Christopher Collins]

On Thu, Apr 20, 2017 at 09:43:12PM -0700, Jacob Rosenthal wrote:
> supervision_timeout = 300 appears to have done it! Any good way to find the
> existing params change just the 1 I actually want to change?

You can use ble_gap_conn_find() to determine the current connection
parameters.

> Maybe nrf51 devices should look up supervision_timeout and either send an
> error code or even request parameter update automatically?
> 
> static int
> bleprph_gap_event(struct ble_gap_event *event, void *arg)
> {
> struct ble_gap_conn_desc desc;
> int rc;
> 
> switch (event->type) {
> case BLE_GAP_EVENT_CONNECT:
> struct ble_gap_upd_params params = {
> .itvl_min = BLE_GAP_ADV_FAST_INTERVAL1_MIN,
> .itvl_max = BLE_GAP_ADV_FAST_INTERVAL1_MAX,
> .supervision_timeout = 300,
> .min_ce_len = BLE_GAP_INITIAL_CONN_MIN_CE_LEN,
> .max_ce_len = BLE_GAP_INITIAL_CONN_MAX_CE_LEN,
> };
> rc = ble_gap_update_params(event->connect.conn_handle, );
> assert(rc == 0);
> return 0;

Careful, you're specifying advertising interval values as connection
interval parameters.  Using ble_gap_conn_find() will take care of this.

Chris

Re: newtmgr image upload nrf51dk disconnects with reason=8

[Christopher Collins]

Hi Jacob,

On Thu, Apr 20, 2017 at 02:21:01PM -0700, Jacob Rosenthal wrote:
> I think the default intervals are here, which should be sufficiently over
> 20ms
> /** 30 ms. */
> #define BLE_GAP_ADV_FAST_INTERVAL1_MIN  (30 * 1000 / BLE_HCI_ADV_ITVL)
>   //48
> 
> /** 60 ms. */
> #define BLE_GAP_ADV_FAST_INTERVAL1_MAX  (60 * 1000 / BLE_HCI_ADV_ITVL).
>   //96
> 
> /** 100 ms. */
> #define BLE_GAP_ADV_FAST_INTERVAL2_MIN  (100 * 1000 / BLE_HCI_ADV_ITVL)
> //160
> 
> /** 150 ms. */
> #define BLE_GAP_ADV_FAST_INTERVAL2_MAX  (150 * 1000 / BLE_HCI_ADV_ITVL)
> //240
> 
> or I can even force to the higher ones in bleprph:
> 
> adv_params.itvl_min = BLE_GAP_ADV_FAST_INTERVAL2_MIN;
> adv_params.itvl_max = BLE_GAP_ADV_FAST_INTERVAL2_MAX;
[...]
> > 
> > In logs, interval advertises as adv_itvl_min=0 adv_itvl_max=0, on
> > connection logs ive seen both itvl=9 and itvl=12, not sure what that maps
> > to yet though..

These are advertising intervals, i.e., how frequently the device sends
out an advertisement, and not relevant here.  BLE has a lot of
"intervals," so I don't fault you for getting confused!

The interval of importance here is the connection interval, i.e., how
frequently the two connected peers attempt communication.  

I think adjusting the connection interval is not the best solution for
the supervision timeout problem.  If you increase the interval, this
might help maintain the connection, but it will also impair response
time between the two devices.  The correct setting to increase, in my
opinion, is the supervision timeout.  By increasing this, you allow the
peripheral device to stay quiet for a longer period of time without the
connection getting dropped.

Regardless of which parameter you change, there are two ways to do it:
1. Configure the central device such that it initiates connections
   with the preferred settings (this is what Vipul suggested in his
   newtmgr diff).

2. Have the peripheral device request a connection parameter update
   after the connection is established.  With NimBLE, you would use
   the ble_gap_update_params() function for this
   
(https://mynewt.apache.org/latest/network/ble/ble_hs/ble_gap/functions/ble_gap_update_params/).

Option 1 is the reliable way to do this.  With option 2, the central may
choose to disregard your requested parameters, so you're really at the
central's mercy.

Chris

Re: newtmgr image upload nrf51dk disconnects with reason=8

[Christopher Collins]

On Wed, Apr 19, 2017 at 04:33:05PM -0700, Jacob Rosenthal wrote:
> On Wed, Apr 19, 2017 at 11:19 AM, marko kiiskila  wrote:
> 
> > Just general comments, I hope I’m not saying things which are
> > too obvious.
> >
> More specific would be even better :) I dont think my gdb is up to par
> 
> Either g_os_run_list or one of the task structures is getting smashed.
> > As you know you tasks beforehand, you can walk through them manually
> > to figure which one it is.
> >
> How do I know the tasks beforehand? I would guess something in imgr_upload
> is corrupting it? So print as that function starts and ends? How do I walk
> through them manually?

You probably only have two tasks ("main" and the BLE controller).  My
first guess would be that the main stack is overflowing, since that is
used by both the host and newtmgr.  An easy way to tell if this is the
case is:

1. Reproduce the crash with gdb running.
2. In gdb type:

x/32wx os_main_stack

The first several values printed should be 0xdeadbeef.  If they are
something else, then you have a stack overflow.  You should increase the
size of the main stack by adjusting the OS_MAIN_STACK_SIZE syscfg
setting (default=1024 words).

> 
> >
> > Usually culprit is stack overflow, so once you find out which task
> > structure
> > is being corrupt, look for the stack just after that in memory.
> >
> > nm output piped to sort is your friend in locating that stack.
> >
> nm output?

The "nm" tool shows you the address and size of each symbol in an elf
file.  I recommend passing the "-S" option to it so that the sizes are
included in the output.

Chris

Re: newtmgr image upload nrf51dk disconnects with reason=8

[Christopher Collins]

Hi Jacob,

On Sun, Apr 16, 2017 at 03:47:24PM -0700, Jacob Rosenthal wrote:
> I got a linux partition and newtmgr working finally, However the upload
> command fails after writing one packet. The logs appear to show
> BLE_HS_ENOTSUP , but I just cant find where that could becoming from! any
> help appreciated :)

Welcome to return code hell :).  The status=8 is actually a Bluetooth
error code, not a NimBLE host error code.  In this case, the controller
belonging to the destination device has indicated a disconnect reason of
8, which translates to "supervision timeout."  In other words, your PC's
controller unexpectedly went silent, so the NimBLE device was forced to
drop the connection.

The GATT library (what newtmgr uses for BLE) is a bit sketchy, so it's
hard to tell who as at fault.  What controller are you using to send the
newtmgr command (e.g., built-in Bluetooth radio)?

Do you have any luck with a smaller newtmgr command (e.g., echo)?  The
image upload command is especially troublesome because it causes the
destination device to perform a flash erase operation.  This causes the
MCU to stall, which can lead to supervision timeouts.  I believe the
newtmgr tool uses fairly lenient connection settings for this reason,
but this could still be the problem, especially on the nRF51.

You could try connecting to the NimBLE device with a known good setup
such as the Lightblue app on OS X or an iphone.  Then, subscribe to the
newtmgr characteristic and try writing a value to the same
characteristic.  If the connection stays up and you get a notification
from the NimBLE device (newtmgr response), then I would suspect
something on the PC side.

Chris

> 
> 
> Jacobs-MacBook-Air:chippd3 jacobrosenthal$ newt target show split-nrf51dk
> targets/split-nrf51dk
> app=@apache-mynewt-core/apps/blesplit
> bsp=@apache-mynewt-core/hw/bsp/nrf51dk
> build_profile=optimized
> loader=@apache-mynewt-core/apps/bleprph
> 
> syscfg=BLE_ACL_BUF_SIZE=128:BLE_LL_CFG_FEAT_LE_ENCRYPTION=0:BLE_SM_LEGACY=0:LOG_LEVEL=0
> 
> 
> [jacobrosenthal@localhost Downloads]$ sudo "$(which newtmgr)" -c ble image
> upload  blesplit.img -t -ldebug
> 2017/04/14 15:02:08 [DEBUG] BLE Connection devaddr:[]
> 2017/04/14 15:02:08 dev: hci0 up
> 2017/04/14 15:02:08 dev: hci0 down
> 2017/04/14 15:02:08 dev: hci0 opened
> 2017/04/14 15:02:08 [DEBUG] State:PoweredOn
> 2017/04/14 15:02:08 [DEBUG] scanning...
> 2017/04/14 15:02:08 [DEBUG] Peripheral Discovered: nimble-bleprph,
> Address:[10 10 10 10 10 10] Address Type:0
> 2017/04/14 15:02:08 [DEBUG] Peripheral Connected
> 2017/04/14 15:02:08 [DEBUG] Newtmgr Service Found
> 2017/04/14 15:02:08 [DEBUG] Newtmgr Characteristic Found
> 2017/04/14 15:02:08 [DEBUG] Writing newtmgr request &{Op:2 Flags:0 Len:99
> Group:1 Seq:0 Id:1 Data:[163 100 100 97 116 97 88 79 60 184 243 150 36 0 0
> 0 32 0 0 0 72 16 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 64 0 32 41 56 2 0 0
> 26 128 243 20 136 128 243 16 136 3 33 24 72 2 104 10 67 2 96 23 72 2 104 10
> 67 2 96 0 26 22 74 22 75 154 66 1 210 1 99 108 101 110 25 16 140 99 111 102
> 102 0]}
> 2017/04/14 15:02:08 [DEBUG] Serializing request &{Op:2 Flags:0 Len:99
> Group:1 Seq:0 Id:1 Data:[163 100 100 97 116 97 88 79 60 184 243 150 36 0 0
> 0 32 0 0 0 72 16 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 64 0 32 41 56 2 0 0
> 26 128 243 20 136 128 243 16 136 3 33 24 72 2 104 10 67 2 96 23 72 2 104 10
> 67 2 96 0 26 22 74 22 75 154 66 1 210 1 99 108 101 110 25 16 140 99 111 102
> 102 0]} into buffer [2 0 0 99 0 1 0 1 163 100 100 97 116 97 88 79 60 184
> 243 150 36 0 0 0 32 0 0 0 72 16 0 0 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 64 0
> 32 41 56 2 0 0 26 128 243 20 136 128 243 16 136 3 33 24 72 2 104 10 67 2 96
> 23 72 2 104 10 67 2 96 0 26 22 74 22 75 154 66 1 210 1 99 108 101 110 25 16
> 140 99 111 102 102 0]
> 2017/04/14 15:02:08 [DEBUG] Tx packet dump:
>   02 00 00 63 00 01 00 01  a3 64 64 61 74 61 58 4f
>  |...c.ddataXO|
> 0010  3c b8 f3 96 24 00 00 00  20 00 00 00 48 10 00 00  |<...$...
> ...H...|
> 0020  12 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00
>  ||
> 0030  00 40 00 20 29 38 02 00  00 1a 80 f3 14 88 80 f3  |.@.
> )8..|
> 0040  10 88 03 21 18 48 02 68  0a 43 02 60 17 48 02 68
>  |...!.H.h.C.`.H.h|
> 0050  0a 43 02 60 00 1a 16 4a  16 4b 9a 42 01 d2 01 63
>  |.C.`...J.K.B...c|
> 0060  6c 65 6e 19 10 8c 63 6f  66 66 00 |len...coff.|
> 
> 2017/04/14 15:02:08 [DEBUG] Write BLE Packet:buf:: c �ddataXO<���$ H @ )8
> �� ��� � ! H h
> C ` H h
> C ` J K�B � clen �coff len::107
> 2017/04/14 15:02:12 [DEBUG] Disconnected%!(EXTRA )
> 
> 
> Logs show its 520, presumably BLE_HS_ENOTSUP???
> 
> 832:[ts=6499968ssb, mod=4 level=0] Disconnection Complete: status=0
> handle=1 reason=8
> 834:[ts=6515592ssb, mod=64 level=1] connection updated; status=520 handle=1
> our_ota_addr_type=0 our_ota_addr=0a:0a:0a:0a:0a:0a our_id_addr_type=0
> our_id_addr=0a:0a:0a:0a:0a:0a peer_ota_addr_type=0
> 

Re: Use of os_error_t and OS_OK

[Christopher Collins]

On Mon, Apr 10, 2017 at 04:42:09PM +0200, Sterling Hughes wrote:
> I don’t think we ever came to agreement, and things are a bit of a 
> mishmash.  Ben brings up a good point.
> 
> Mynewt wide, in my view:
> 
> * os_error is a relic, and sys/defs codes should be used.
> 
> * All functions should return “int” and not “os_error_t” or a 
> specific error type.
> 
> * 0 and -1 are SYS_EOK and SYS_EUNKNOWN (new value) respectively, and 
> can be safely returned as numbers not defines.
> 
> * For other errors, the SYS_* equivalents should be returned.
> 
> Thoughts?

Sounds good to me.  I particularly like using 0 rather than SYS_EOK.
It's completely subjective, but I think it makes the code easier to
read.

Chris

Re: mynewt, build on ubuntu

[Christopher Collins]

Thanks Filipe.  I just pushed a fix for this issue.  The setting type
that caused this problem (interrupt_priority) was broken and unused
anyway.  I thought we had already removed it, but obviously not!  The
fix is to remove this setting type.

I believe the latest on master should build on a 32-bit system.

Chris

On Sat, Apr 08, 2017 at 07:06:24PM +0100, F Campos wrote:
> Hi,
> when building the mynewt on ubuntu I have an error on go builder:
> 
> *syscfg/syscfg.go:965: constant 4294967295 overflows int*
> 
> Seems that the SYSCFG_INTERRUPT_PRIO_MAX is too big for the variable "int"
> 
> 
> 
> *Linux Version:*
> lsb_release -a
> Distributor ID: Ubuntu
> Description: Ubuntu 14.04.4 LTS
> Release: 14.04
> Codename: trusty
> 
> 
> *Go version:*
> go version
> go version go1.7 linux/386
> 
> 
> Filipe

Develop branch deprecated

[Christopher Collins]

Hello all,

I have just merged develop to master (as you no doubt noticed if you are
subscribed to the commits list!).  From now on, all commits should go
into separate feature branches; develop is deprecated and should not be
used anymore.  The Mynewt branching policy is defined here:
https://cwiki.apache.org/confluence/display/MYNEWT/Release+and+Support+Policy

There is a related annoyance that contributors will need to deal with:
existing pull requests.  All the existing PRs have a target branch of
"develop"; these will need to be updated to have a target branch of
"master".  Sorry for the hassle, everyone.

All questions and concerns are welcome.

Chris

Re: Problem loading images to nRF52DK board

[Christopher Collins]

Hi Greg,

On Tue, Apr 04, 2017 at 06:48:31PM +, Greg Foringer wrote:
> Hello,
> 
> I've spent about 16 hours trying to get the nrf52_boot and blehci image
> flashed to my board. I'm hoping someone has some insight about my problem.
> 
> I have installed all the SEGGER JLink tools and drivers (v6.14c) on both my
> OSX host and my Ubuntu 16.04 guest. When using JLinkExe, it can connect to
> the board over USB. I'm using the latest 1.0.0 version of newt. I've
> followed the instructions to build the nrf52_boot bootloader for my nRF52DK
> board (PCA10040 v1.1.0) as well as the blehci app.

[...]

> gdb_cmds:2: Error in sourced command file: localhost:: Connection timed
> out.

Hmm, it looks like JLinkGDBServer isn't staying up (or isn't getting
started at all).  Probably unlikely, but still worth asking: could you
have an instance of JLinkExe already running on that machine?  If any
JLink tool is already running, a second invocation will fail.

One way to debug this is to manually do what newt does and see where
the process fails:

In terminal 1:
JLinkGDBServer -device nRF52 -speed 4000 -if SWD -port 
-singlerun -reset

In terminal 2:
arm-none-eabi-gdb

(gdb) mon reset
Resetting target

(gdb) restore 
/Users/ccollins/repos/mynewt/core/bin/targets/boot-nrf52dk/app/apps/boot/boot.elf.bin
 binary 0
Restoring binary file 
/Users/ccollins/repos/mynewt/core/bin/targets/boot-nrf52dk/app/apps/boot/boot.elf.bin
 into memory (0x0 to 0x23e0)

(of course you will need to adjust the path of your boot binary).

You can also load the image into slot 0:

(gdb) restore 
/Users/ccollins/repos/mynewt/core/bin/targets/blehci-nrf52dk/app/apps/blehci/blehci.img
 binary 0x8000

Chris

Re: Adding platform specific API to get public and/or random static address

[Christopher Collins]

On Sat, Apr 01, 2017 at 09:53:03AM +0200, Marcel Holtmann wrote:
> > Some things about writing apps and the BLE spec:
> > 1) I realize that it is the host that tells the controller the
> > random address to use. The controller will NOT automatically use the
> > random address from ble_hw_get_static_addr(). That API will be added
> > as a convenience so that the app developer does not have to generate
> > their own. If the app wants to use this random address it needs to
> > tell the controller to use it using LE_Set_Random_Addr.
> > 
> > 2) Regarding the public device address. We have an app called
> > bletiny that can set the public device address I think. If the above
> > gets approved we are going to remove g_dev_addr from the code; it
> > will be kept in the controller and not available globally. The
> > Zephyr project is considering adding vendor specific HCI commands,
> > one of which is “set public device address”. I think if we go with
> > the above approach we should add this vendor specific command and
> > that should be the way an app can set the public device address if
> > it so chooses.
> 
> The public BD_ADDR needs to be inside the controller before the call
> of HCI_Reset. Otherwise all sort of assumptions on HCI break. Until
> then the HCI_Read_BD_ADDR has to return 00:00:00:00:00:00 to indicate
> the controller has no public address. Switching the BD_ADDR mid-flight
> with a backdoor is going to fail in the most funny ways left and
> right.

The bletiny app is a sandbox test tool, and it does some things that a
more robust application shouldn't do.  One such underhanded thing it
does is change its own public address whenever the user requests it.  It
does this by simply overwriting the global public address byte array and
hoping for the best.  In practice, I've never seen anything funny
happen, either to the left or the right :), but this is certainly not
guaranteed to work.  Also, this won't work at all unless bletiny is
running on a combined host-controller, since that is the only occasion
in which the host has access to the public address global variable.

I don't want to speak for Will, but my guess is he just looked at what
existing code accesses the public address global to determine the scope
of the proposed API.  My understanding from reading his email is that
bletiny is doing something sketchy, and that the proposed API won't
support this particular use case.

Chris

Re: Adding platform specific API to get public and/or random static address

[Christopher Collins]

On Fri, Mar 31, 2017 at 03:49:05PM -0700, will sanfilippo wrote:
> Hello:
> 
> There has been some discussion of this already on the list but nothing has 
> been done yet so I wanted to resurrect the conversation with some proposals.
> 
> What we are trying to do here is the following:
> 1) Have the controller get a public device address without it being hardcoded.
> 2) Have the ability to read a chip-specific random static address if the chip 
> has one programmed.
> 
> The proposal is the following:

> 1) Add two new API. These will be platform specific and will be placed
> in the ble_hw.c file:

> /* These API will return -1 if no address available. If available, will
> return 0
> and will place the address in *addr */
> int ble_hw_get_public_addr(ble_addr_t *addr)
> int ble_hw_get_static_addr(ble_addr_t *addr)

[...]

That sounds good to me.  This covers all the use cases I can think of.

As you mentioned, Bluetooth is somewhat asymmetric regarding public and
random addresses.  The controller is in charge of the public address
while the host is in charge of the static random address.

With the API you proposed, I think the workflow would look something
like this:

1. At init time, controller calls ble_hs_get_public_addr().  If this
call yields an address, the controller configures itself to use it.

2. If app wants a static random address, it calls
ble_hw_get_static_addr().  If a random address is available, the
application configures the host to use it with a call to
ble_hs_id_set_rnd().

Does that sound about right?

In thinking about this, I realized the host interface is missing
something.  There is currently no way for an application to ask the host
for its public address.  An application may want to know this to
determine if it should configure a random address (or just for
reporting purposes).  The host does know its own public address--it gets
it from the controller at startup--it just doesn't expose it to the
application.

Chris

Re: Newtmgr over BLE not Working

[Christopher Collins]

Hi Lm,

On Sun, Mar 26, 2017 at 07:45:37AM +, Lm Chew wrote:
> I also tried setting up a nrf52 DK with blehci app + BlueZ as explained in 
> this tutorial(with a few changes).
> 
> - http://mynewt.apache.org/latest/os/tutorials/blehci_project/
> 
> - Use this to connect with btattach instead: sudo btattach -B /dev/ttyACM0 -S 
> 100 -P h4
> 
> - Skip the btmgmt step since in BlueZ 5.44 we no longer need to set the 
> static-address manually and newtmgr will handle the rest.
> 
> - Used the nrf52 DK onboard USB to UART bridge, did not use a FTDI USB TTL 
> Serial Cable.
> 
> 
> But unfortunately I can't get it to work either, in fact it is one step 
> backward.

When you use the USB virtual COM port on an nRF board, there are two
things you will want to do:

1. Disable mass storage device capability on your board (from JLinkExe):

MSDDisable

2. Reduce the BLE_ACL_BUF_SIZE setting to 128 (default=255) (in your
target's syscfg.yml file):

syscfg.vals:
BLE_ACL_BUF_SIZE: 128

Both of these changes prevent the virtual COM port's buffer from
overflowing.

For 1: nRF boards normally reserve a large portion of the buffer
for mass storage device functionality.  By disabling this functionality,
the entire buffer is made available to UART opertaions.  

For 2: The buffer can overflow if the host floods the controller with
large ACL data packets, especially at high baud rates.  Reducing the
maximum transmission size down to 128 solved this problem for me.

I don't know if these will solve your problem, though.  If you still
have issues, could you please send the entire btmon log?

Thanks,
Chris

Re: Problem to use newtmgr through ble

[Christopher Collins]

On Fri, Mar 17, 2017 at 08:36:01AM -0300, Fabio Utzig wrote:
> On Fri, Mar 17, 2017, at 08:24 AM, then yon wrote:
> > Dear Chris,
> > 
> > When i put sudo newtmgr -c BT01 stat list, it gave me sudo: newtmgr: 
> > command not found.
> 
> It looks like sudo is not preserving your user path, you could try:
> 
> sudo "PATH=$PATH" newtmgr -c BT01 stat list

Hmm, I don't think you'll be able to set environment variables unless
you execute a new shell instance in sudo (e.g., sudo bash -c ...), which
is painful.

This might be the easiest way forward:

sudo "$(which newtmgr)" -c BT01 stat list

Chris

Re: Debugging blecent application on nrf52dk

[Christopher Collins]

Hi Pritish,

On Thu, Mar 16, 2017 at 01:50:12PM -0700, Pritish Gandhi wrote:
> Hi All,
> I'm trying to run blecent on an nrf52dk and am running the bleprph
> application on another BLE module (stm32f4discovery talking to a broadcom
> BLE core). Anyways, when try to run blecent it seems like I successfully
> connect to the peripheral and are able to discover it, however after that
> the connection seems to be timing out and then am never able to discover
> the peripheral again.

[...]

Hmm, that is odd, indeed.  The disconnect reason codes you are seeing
are mapped as follows:

546 - LMP RESPONSE TIMEOUT / LL RESPONSE TIMEOUT
520 - CONNECTION TIMEOUT

I'm afraid I don't have any ideas at the moment.  Could you please
clarify the setup you are using?  Here is my understanding:

Device A: blecent on nRF52dk (combined host-controller)
Device B:
* bleprph on stm32f4discovery (host-only)
* broadcom controller

Is that correct?  If so, I assume the host and controller on device B
communicate via UART?

Thanks,
Chris

> 
> 1) Connected and Discovered the bleprph:
> 
> 37493:[ts=292914004ssb, mod=4 level=1] GAP procedure initiated: discovery;
> own_addr_type=0 filter_policy=0 passive=1 limited=0 filter_duplicates=1
> duration=forever
> 
> 37503:[ts=292992124ssb, mod=4 level=1] GAP procedure initiated: connect;
> peer_addr_type=0 peer_addr=aa:aa:aa:aa:aa:aa scan_itvl=16 scan_window=16
> itvl_min=24 itvl_max=40 latency=0 supervision_timeout=256 min_ce_len=16
> max_ce_len=768 own_addr_ty
> 
> 37517:[ts=293101556ssb, mod=64 level=1] Connection established
> 
> 37519:[ts=293117180ssb, mod=4 level=1] GATT procedure initiated: discover
> all services
> 
> 37588:[ts=293656208ssb, mod=4 level=1] GATT procedure initiated: discover
> all characteristics; start_handle=1 end_handle=11
> 
> 37627:[ts=293960876ssb, mod=4 level=1] GATT procedure initiated: discover
> all characteristics; start_handle=12 end_handle=15
> 
> 37658:[ts=294203112ssb, mod=4 level=1] GATT procedure initiated: discover
> all characteristics; start_handle=16 end_handle=19
> 
> 37684:[ts=294406224ssb, mod=4 level=1] GATT procedure initiated: discover
> all characteristics; start_handle=20 end_handle=32
> 
> 37722:[ts=294703080ssb, mod=4 level=1] GATT procedure initiated: discover
> all characteristics; start_handle=33 end_handle=65535
> 
> 37761:[ts=295007812ssb, mod=4 level=1] GATT procedure initiated: discover
> all descriptors; chr_val_handle=14 end_handle=15
> 
> 37774:[ts=295109368ssb, mod=4 level=1] GATT procedure initiated: discover
> all descriptors; chr_val_handle=18 end_handle=19
> 
> 37786:[ts=295203112ssb, mod=4 level=1] GATT procedure initiated: discover
> all descriptors; chr_val_handle=24 end_handle=25
> 
> 37799:[ts=295304668ssb, mod=4 level=1] GATT procedure initiated: discover
> all descriptors; chr_val_handle=29 end_handle=30
> 
> 37812:[ts=295406224ssb, mod=4 level=1] GATT procedure initiated: discover
> all descriptors; chr_val_handle=37 end_handle=65535
> 
> 37825:[ts=295507780ssb, mod=64 level=3] Service discovery complete;
> status=0 conn_handle=1
> 
> 2) Read/Write/Subscribe for notifications. Finally fails with reason=546
> 
> 37827:[ts=295523404ssb, mod=4 level=1] GATT procedure initiated: read;
> att_handle=22
> 
> 37829:[ts=295539028ssb, mod=4 level=1] GATT procedure initiated: write;
> att_handle=32 len=2
> 
> 37832:[ts=295562464ssb, mod=4 level=1] GATT procedure initiated: write;
> att_handle=30 len=2
> 
> 37851:[ts=295710892ssb, mod=64 level=1] Read complete; status=0
> conn_handle=1 attr_handle=22 value=
> 
> 37857:[ts=295757764ssb, mod=64 level=1] Write complete; status=0
> conn_handle=1 attr_handle=32
> 
> 37863:[ts=295804636ssb, mod=64 level=1] Subscribe complete; status=0
> conn_handle=1 attr_handle=30
> 
> 42637:[ts=333101556ssb, mod=64 level=1] disconnect; reason=546
> 
> 
> 3) Once it disconnects, blecent gets stuck in this loop of trying to
> discover, but the discovery always fails:
> 
> 42638:[ts=333109368ssb, mod=4 level=1] GAP procedure initiated: discovery;
> own_addr_type=0 filter_policy=0 passive=1 limited=0 filter_duplicates=1
> duration=forever
> 
> 42973:[ts=335726516ssb, mod=4 level=1] GAP procedure initiated: connect;
> peer_addr_type=0 peer_addr=aa:aa:aa:aa:aa:aa scan_itvl=16 scan_window=16
> itvl_min=24 itvl_max=40 latency=0 supervision_timeout=256 min_ce_len=16
> max_ce_len=768 own_addr_ty
> 
> 42982:[ts=335796824ssb, mod=64 level=1] Connection established
> 
> 42983:[ts=335804636ssb, mod=4 level=1] GATT procedure initiated: discover
> all services
> 
> 43020:[ts=336093744ssb, mod=64 level=3] Error: Service discovery failed;
> status=7 conn_handle=1
> 
> 43022:[ts=336109368ssb, mod=4 level=1] GAP procedure initiated: terminate
> connection; conn_handle=1 hci_reason=19
> 
> 43025:[ts=336132804ssb, mod=64 level=1] disconnect; reason=520
> 
> 43027:[ts=336148428ssb, mod=4 level=1] GAP procedure initiated: discovery;
> own_addr_type=0 filter_policy=0 passive=1 limited=0 filter_duplicates=1

Re: Problem to use newtmgr through ble

[Christopher Collins]

Hi Then,

I have found that I need to run newtmgr as root when using ble (e.g.,
sudo newtmgr ...)

Chris

On Thu, Mar 16, 2017 at 05:22:01PM +0800, then yon wrote:
> Dear Support,
> 
> Anyone have answer on this?
> 
> Some updates on the newtmgr setting:
> 
> Connection profiles: BT01: type=ble, connstring='nimble-bleprph'
> 
> Whenever i sent newtmgr command i get the following:
> 
> 2017/03/16 17:11:35 dev: hci0 up 2017/03/16 17:11:35 dev: hci1 up Error: 
> no supported devices available
> 
> I'm using V2TEC Bluetooth dongle (Bus 001 Device 016: ID 0a12:0001 
> Cambridge Silicon Radio, Ltd Bluetooth Dongle (HCI mode))
> 
> Thank you.
> 
> Regards,
> 
> Then Yoong Ze
> 
> 
> On 10/3/2017 6:59 PM, then yon wrote:
> > Dear Support,
> >
> > After the being success to do image upgrade with slinky app, now i
> > moving forward to image upgrade through ble with bleprph example.
> >
> > With the target.yml as below:
> > ### Target: targets/split_slinky
> > target.app: "@apache-mynewt-core/apps/splitty"
> > target.bsp: "@apache-mynewt-core/hw/bsp/nrf52dk"
> > target.build_profile: "optimized"
> > target.loader: "@apache-mynewt-core/apps/bleprph"
> > target.syscfg: "BLE_LL_CFG_FEAT_LE_ENCRYPTION=0:BLE_SM_LEGACY=0"
> >
> > The problem is i do not get any newtmgr response in serial with this
> > example; and i can't detect any BT packet from bleprph example.
> >
> > if i changed the target.yml as below; i can detect bleprph advertising
> > packet:
> > ### Target: targets/split_slinky
> > target.bsp: "@apache-mynewt-core/hw/bsp/nrf52dk"
> > target.build_profile: "optimized"
> > target.app: "@apache-mynewt-core/apps/bleprph"
> > target.syscfg: "BLE_LL_CFG_FEAT_LE_ENCRYPTION=0:BLE_SM_LEGACY=0"
> >
> >
> > The command i used to detect ble advertising packet is "sudo hcitool
> > lescan --duplicates &"
> >
> >
> > Thank you.
> >
> > Regards,
> >
> > Then Yoong Ze
> >
> 

Re: [VOTE] Release Apache Mynewt 1.0.0-incubating-rc1

[Christopher Collins]

On Thu, Mar 09, 2017 at 10:24:32AM -0800, marko kiiskila wrote:
> [X] +1 Release this package

+1 (Binding)

Chris

Re: building newt using go

[Christopher Collins]

On Thu, Mar 09, 2017 at 07:52:58AM +, David Brown wrote:
> I would suggest we remove this particular dependency. Although it is nice
> to have symbolic names, go 1.6 is the latest version that is in a LTS
> ubuntu release, so I suspect it is the easiest version for a lot of people
> to have available.
> 
> The symbol in question is just an integer constant.

[io.SeekCurrent]

That is a good idea.  I didn't realize that symbol got added in 1.7.  I
just used it because the Go docs said it is a replacement for a
deprecated symbol.

Chris

Re: MyNewt: NimBLE: Question regarding large file transfer over Bluetooth

[Christopher Collins]

Hi Pritish,

On Tue, Mar 07, 2017 at 03:15:49PM -0800, Pritish Gandhi wrote:
> Hi All,
> Disclaimer: I don't know much about Bluetooth so I might sound like a novice
> 
> I've been thinking about writing an application to transfer an OTA image
> over Bluetooth and am wondering what would be the best way to go about it.
> I was looking at the generic Object Transfer Services in the Bluetooth Spec:
> https://www.bluetooth.com/specifications/gatt/viewer?attributeXmlFile=org.bluetooth.service.object_transfer.xml
> 
> And it seems to suggest that this service uses a separate L2CAP connection.
> I wonder whether NimBLE supports this or any other file transfer protocol
> over Bluetooth.
> Has anyone tried this or some other means of doing an OTA over Bluetooth?

I'm not familiar with the object transfer service, but it looks like it
would work pretty well for this.  Łukasz has added support for
connection oriented channels to nimble, so I think the required
functionality is all there.  This service looks rather complicated,
though, so there may be a fair amount of work involved in implementing
it.

Alternatively, Mynewt already supports file transfer using the newtmgr
protocol (NMP).  NMP is a simple request-response protocol described in
more detail here:
http://mynewt.apache.org/latest/os/modules/devmgmt/newtmgr/

This works today, but it is not particularly well suited to large data
transfers over BLE.  The need to wait for a response to each file chunk
prevents the kind of throughput that a BLE link can support.

Chris

NMP over OIC

[Christopher Collins]

Hello all,

I encountered some issues while using the newtmgr protocol (NMP) over
OIC.  In this email, I will call this protocol stack (NMP over OIC) OMP.
The issues, as I see them, are:

1. Parts of NMP header missing from OMP responses.  This causes problems
when multiplexing OMP among many target devices.

2. Parts of the NMP header are scattered in a few different places.
This works, but it makes it difficult to debug packet traces.

3. NMP errors get lost in translation.  Instead of an NMP message with
an error code, the OMP server sends back a generic "Bad Request" OIC
message.

I have addressed these issues in the following proposal:
https://issues.apache.org/jira/browse/MYNEWT-647

If you're like me, you might not bother reading something unless it is
reproduced in the email body :).  The contents of the aforementioned jira
ticket are below.

Comments welcome.

Thanks,
Chris

--

* Current Scheme

*** Requests

The NMP op is indicated by the OIC op:
NMP read <=> OIC get
NMP write <=> OIC put

The NMP group and ID are indicated as CoAP URI Query options:

gr=
id=

The remaining NMP header fields, seq and flags, are not present.

The NMP payload is the entire CoAP request body. This is identical to the body
of a plain NMP request.

*** Responses

The NMP header is not present. The NMP op is indicated in the payload (see
below), but other header fields cannot be inferred.

Payload consists of a single CBOR key-value pair. For read responses, the key
is "r"; for write responses, the key is "w". The value is a second CBOR map
containing the actual NMP response fields.

Errors encountered during processing of NMP requests are reported via OIC error
responses (bad request, internal server error).

* Proposed Scheme

*** Requests

* The OIC op is always the same: put.

* No URI Query CoAP options.

* The NMP header is included in the payload as a key-value pair (key="_h").
  This pair is in the root map of the request and is a sibling of the other
  request fields. The value of this pair is the big-endian eight-byte NMP
  header with a length field of 0.

*** Responses

* As with requests, the NMP header is included in the payload as a
  key-value pair (key="_h").

* No "r" or "w" field. The response fields are inserted into the root map
  as a sibling of the "_h" pair.

* Errors encountered during processing of NMP requests are reported
  identically to other NMP responses (embedded NMP response).

*** Notes

* Keys that start with an underscore are reserved to the OIC manager
  protocol (e.g., "_h"). NMP requests and responses must not name any of
  their fields with a leading underscore.

Re: issue with FCB reboot logs

[Christopher Collins]

Hi Szymon,

On Tue, Feb 28, 2017 at 03:51:51PM +0100, Szymon Janc wrote:
> Hi,
> 
> I noticed that some applications (eg bleprhp) enable FCB logging in
> syscfg.yml by default.
> When flashing such sample on nRF52DK I get assertion [1] before even
> main() is reached.
> And if I reset a board I get loop of assert and unhandled interrupt
> exception [2].
> 
> Disabling FCB makes sample works fine. Do I need to do any magic to
> get it working
> with FCB enabled?

My guess is that there is something else written to the reboot log area.
I believe the reboot log package asserts that it initializes
successfully.  Do you know what code is at the indicated address
(0x1a393)?

This would be the case if it has been a long time since you erased your
device's flash.  The flash map changed in an non-backwards-compatible
way between 0.9.0 and 1.0.0-b1.

> Also, should those be enabled by default in samples in first place?

Yes, that is probably a good idea IMO.

Chris

> 
> [1]
> 1:Assert @ 0x1a393
> 
> [2]
> 1:Assert @ 0x1a38b
> 1:Unhandled interrupt (2), exception sp 0x2ba0
> 1: r0:0x  r1:0x  r2:0x8000  r3:0xe000ed00
> 1: r4:0x0001a38b  r5:0x  r6:0x  r7:0x
> 1: r8:0x  r9:0x r10:0x r11:0x
> 1:r12:0x  lr:0x8875  pc:0x8884 psr:0x61000200
> 1:ICSR:0x00421802 HFSR:0x CFSR:0x
> 1:BFAR:0xe000ed38 MMFAR:0xe000ed34
> 1:Assert @ 0x1a38b
> 1:Unhandled interrupt (2), exception sp 0x2ba0
> 1: r0:0x  r1:0x  r2:0x8000  r3:0xe000ed00
> 1: r4:0x0001a38b  r5:0x  r6:0x  r7:0x
> 1: r8:0x  r9:0x r10:0x r11:0x
> 1:r12:0x  lr:0x8875  pc:0x8884 psr:0x61000200
> 1:ICSR:0x00421802 HFSR:0x CFSR:0x
> 1:BFAR:0xe000ed38 MMFAR:0xe000ed34
> 
> 
> -- 
> pozdrawiam
> Szymon K. Janc

Re: Package custom build steps

[Christopher Collins]

Hi Simon,

On Sun, Feb 26, 2017 at 01:31:39PM -0800, Simon Ratner wrote:
> Hi devs,
> 
> I have a package that requires a custom build step (generating intermediate
> files). Any recommendations on how to best handle this?
> 
> Currently I am doing the custom step from my top-level makefile, before
> invoking `newt build`, but that doesn't seem ideal.

I don't think there is currently any better way than what you are
currently doing (generating the files via make).  This is definitely
something that is missing from newt.

It might take some thought to get something like this right.  For now
I'll just throw an idea out there:

Two new settings in the target.yml file:
o target.pre_run
o target.post_run

Each specifies a sequence of executable pathnames.  The pre_run commands
get executed before newt builds anything; the post_run commands are
exected after everything gets built.  If any executable fails, the build
fails.

Chris

Re: BLE HCI support on NRF52DK

[Christopher Collins]

Hi Alan,

On Sat, Feb 25, 2017 at 12:21:15AM +, Alan Graves wrote:
> Well my conclusion is that the HCI mode is frustrating at best to use on the 
> BlueZ stack!  
> 
> In my initial attempts to talk to the controller I use this command:
> $ sudo btmgmt --index 1
> [sudo] password for alan: 
> [hci1]#
> 
> First thing I noticed is that the MyNewt HCI mode tutorial says to set the 
> static address:
> [hci1]# static-addr cc:00:00:00:00:00
> Set static address failed with status 0x0b (Rejected)

I believe you can only configure an address while the controller is
"powered off" (scare quotes because the board doesn't actually have to
be powered off, BlueZ just has to consider it to be in the powered off
state).  Try the following from btmgmt:

power off
static-addr cc:00:00:00:00:00
power on

> This doesn't work on my system. I'm not sure what the problem is, but
> perhaps the initial BT addr 00:00:00:00:00:00 is a clue?

The all-zero address indicates that your controller is not configured
with a public address.  With no public address and no random address
you are pretty restricted in what you can do.

Hopefully this resolves some of the issues you're seeing.  I'm afraid my
knowledge of BlueZ is pretty poor, so I can't say if there is any issue
with the C program

Chris

Re: image upload error 2

[Christopher Collins]

On Fri, Feb 24, 2017 at 09:39:37AM +0800, then yon wrote:
> Dear Chris,
> 
> i've double enter on each line but it still mix up into 1 line, most 
> probably is my email sw problem.
> 
> Below is all the step i did:
> 
> 1. By using slinky app(no changes); build a split_slinky target.

Stop right there :).  What are you using for a loader and what are you
using for an app?

Chris

Re: image upload error 2

[Christopher Collins]

On Thu, Feb 23, 2017 at 06:05:44PM +0800, then yon wrote:
> Dear Chris,
> 
> Yes, i've followed the step you given; but after step 2 it gave me 
> "Error: Timeout reading from serial connection".
> 
> It only recover after i reset nrf52dk power and it show the result i 
> sent to you previously.
> 
> And when i tried to do step 1 again, it gave me "Error:  rc=6"

Is there anything you can do to preserve the newtmgr output in your
emails?  For some reason, the entire newtmgr response gets joined into a
single line, making it difficult to read.  I have reformatted the
responses below.

$ newtmgr -c mynewt_serial image list
Images:
  slot=0
version: 1.0.0
bootable: true
flags: active confirmed
hash: 6d1b9bb73f9cf223145a28e4e0d7702a1b2aaf60a7a6d66497e02a841cd72c8c
  slot=1
version: 1.0.0
bootable: false
flags:
hash: 0fd6f020a3ff70d4cd150e6233185c6e9b8b93f60af97fb67bc728daa6573f0b
Split status: matching

The empty flags field in slot 1 indicates that the device is in
loader-only mode (i.e., only slot 0 is running).  It appears you have
completed step 3.  With slot 1 unused, you can overwrite it with a new
app.

$ newtmgr -c mynewt_serial image test 
0fd6f020a3ff70d4cd150e6233185c6e9b8b93f60af97fb67bc728daa6573f0b
Images:
  slot=0
version: 1.0.0
bootable: true
flags: active confirmed
hash: 6d1b9bb73f9cf223145a28e4e0d7702a1b2aaf60a7a6d66497e02a841cd72c8c
  slot=1
version: 1.0.0
bootable: false
flags: pending
hash: 0fd6f020a3ff70d4cd150e6233185c6e9b8b93f60af97fb67bc728daa6573f0b
Split status: matching

The "test" command does not come until step 5.  What about step 4,
uploading the new loader?  By testing the original app image, you will
just put the device back into its original state.

$ newtmgr -c mynewt_serial reset
Done

$ newtmgr -c mynewt_serial image list  ^C (cause it hang here indefinitely 
so i cancel it)

It seems like your app (slot 1) doesn't support newtmgr or lacks a
newtmgr transport.

$ newtmgr -c mynewt_serial image upload splitty.img
Error: Timeout reading from serial connection
[...]

Which procedure are you following?  In the split upgrade procedure (the
one I pasted in my previous email), you only upload after issuing the
"image confirm" command.

Can you please do the following:

1. Tell me something about the target you are upgrading.  What loader
and app combo are you using?  From the above, it appears your app may
not support newtmgr.

2. Start from step 1 in the procedure I quoted, please indicate which
step you are currently on, and don't skip any steps :).

Thanks,
Chris

> 
> $ newtmgr -c mynewt_serial image list Images:  slot=0  version: 1.0.0 
> <http://1.0.0.0/> bootable: true  flags: active confirmed  hash: 
> 6d1b9bb73f9cf223145a28e4e0d7702a1b2aaf60a7a6d66497e02a841cd72c8c  slot=1 
>   version: 1.0.0 <http://1.0.0.0/> bootable: false  flags: hash: 
> 0fd6f020a3ff70d4cd150e6233185c6e9b8b93f60af97fb67bc728daa6573f0b  Split 
> status: matching $ newtmgr -c mynewt_serial image test 0fd 
> 6f020a3ff70d4cd150e6233185c6e9b8b93f60af97fb67bc728daa6573f0b  Images: 
>   slot=0  version: 1.0.0 <http://1.0.0.0/> bootable: true  flags: active 
> confirmed  hash: 
> 6d1b9bb73f9cf223145a28e4e0d7702a1b2aaf60a7a6d66497e02a841cd72c8c  slot=1 
>   version: 1.0.0 <http://1.0.0.0/> bootable: false  flags: pending 
>   hash: 0fd6f020a3ff70d4cd150e6233185c6e9b8b93f60af97fb67bc728daa6573f0b 
>   Split status: matching $ newtmgr -c mynewt_serial reset  Done $ 
> newtmgr -c mynewt_serial image list  ^C (cause it hang here indefinitely 
> so i cancel it) $ newtmgr -c mynewt_serial image upload splitty.img 
> <http://plitty.img/>Error: Timeout reading from serial connection 
>   upload - Upload image to target  Usage:  newtmgr image upload [flags] 
>   Examples:  newtmgr -c olimex image upload  olimex image upload bin/slinky_zero/apps/slinky.img 
> <http://slinky.img/> Global Flags:  -c, --conn string connection profile 
> to use.  -l, --loglevel string log level to use (default INFO.) 
> <http://info.%29/>(default "info")  -t, --trace print all bytes 
> transmitted and received
> 
> 
> Thank you.
> 
> Regards,
> 
> Then Yoong Ze
> 
> 
> On 23/2/2017 12:44 AM, Christopher Collins wrote:
> > Hi Then,
> >
> > On Wed, Feb 22, 2017 at 02:11:36PM +0800, then yon wrote:
> >> Dear Support,
> >>
> >> I'm followed the tutorial below:
> >> https://mynewt.apache.org/latest/os/modules/split/split/
> >>
> >> But i'm stuck at image upload step; it gave me the following error:
> > [...]
> >> Error: Target error: 2
> > Error 2 i

Re: newtmgr over serial hanging

[Christopher Collins]

Hi Jacob,

On Tue, Feb 21, 2017 at 09:10:08PM -0700, Jacob Rosenthal wrote:
[...]
> I get two distinct hangs to make things more weird.
> 
> Jacobs-MacBook-Air:~ jacobrosenthal$ newtmgr image list -cserial1 -t -ldebug
> 2017/02/21 20:36:03 [DEBUG] Writing newtmgr request &{Op:0 Flags:0 Len:0
> Group:1 Seq:0 Id:0 Data:[]}
> 2017/02/21 20:36:03 [DEBUG] Serializing request &{Op:0 Flags:0 Len:0
> Group:1 Seq:0 Id:0 Data:[]} into buffer [0 0 0 0 0 1 0 0]
> 2017/02/21 20:36:03 [DEBUG] Tx packet dump:
>   00 00 00 00 00 01 00 00   ||
> 
> 2017/02/21 20:36:03 [INFO] Outgoing:
>   06 09 |..|
> 
> 2017/02/21 20:36:03 [DEBUG] Writing [6 9] to data channel
> 2017/02/21 20:36:03 [INFO] Outgoing:
>   41 41 6f 41 41 41 41 41  41 41 45 41 41 44 63 77
>  |AAoAAAEAADcw|
> 
> 2017/02/21 20:36:03 [DEBUG] Writing [65 65 111 65 65 65 65 65 65 65 69 65
> 65 68 99 119] to data channel
> 2017/02/21 20:36:03 [INFO] Outgoing:
>   0a|.|
> 
> 2017/02/21 20:36:03 [DEBUG] Writing [10] to data channel

It looks like the board isn't responding at all.  There are a few things
I would check:

* UART is properly configured in your app with the expected settings
(115200, no flow control).

* The mgmt/newtmgr package gets included in your target.

* An appropriate newtmgr transport gets included in your target (either
mgmt/newtmgr/transport/nmgr_shell or mgmr/newtmgr/transport/nmgr_uart)

The following newtmgr commands would be useful for verifying the above:
newt target dep 
newt target revdep 
newt target config show 

My suspicion is that your target doesn't include a newtmgr transport.

> Jacobs-MacBook-Air:~ jacobrosenthal$ newtmgr image list -cserial1 -t -ldebug
> 2017/02/21 21:07:18 [DEBUG] Writing newtmgr request &{Op:0 Flags:0 Len:0
> Group:1 Seq:0 Id:0 Data:[]}
> 2017/02/21 21:07:18 [DEBUG] Serializing request &{Op:0 Flags:0 Len:0
> Group:1 Seq:0 Id:0 Data:[]} into buffer [0 0 0 0 0 1 0 0]
> 2017/02/21 21:07:18 [DEBUG] Tx packet dump:
>   00 00 00 00 00 01 00 00   ||
> 
> 2017/02/21 21:07:18 [INFO] Outgoing:
>   06 09 |..|
> 
> 2017/02/21 21:07:18 [DEBUG] Writing [6 9] to data channel
> 2017/02/21 21:07:18 [INFO] Outgoing:
>   41 41 6f 41 41 41 41 41  41 41 45 41 41 44 63 77
>  |AAoAAAEAADcw|
> 
> 2017/02/21 21:07:18 [DEBUG] Writing [65 65 111 65 65 65 65 65 65 65 69 65
> 65 68 99 119] to data channel
> 2017/02/21 21:07:18 [INFO] Outgoing:
>   0a|.|
> 
> 2017/02/21 21:07:18 [DEBUG] Writing [10] to data channel
> 2017/02/21 21:07:18 [INFO] Incoming:
>   31 3a 5b 74 73 3d 37 38  31 32 73 73 62 2c 20 6d  |1:[ts=7812ssb,
> m|
> 0010  6f 64 3d 34 20 6c 65 76  65 6c 3d 31 5d 20 47 41  |od=4 level=1]
> GA|
> 0020  50 20 70 72 6f 63 65 64  75 72 65 20 69 6e 69 74  |P procedure
> init|
> 0030  69 61 74 65 64 3a 20 61  64 76 65 72 74 69 73 65  |iated:
> advertise|
> 0040  3b 20 64 69 73 63 5f 6d  6f 64 65 3d 30 20 61 64  |; disc_mode=0
> ad|
> 0050  76 5f 63 68 61 6e 6e 65  6c 5f 6d 61 70 3d 30 20
>  |v_channel_map=0 |
> 0060  6f 77 6e 5f 61 64 64 72  5f 74 79 70 65 3d 30 20
>  |own_addr_type=0 |
> 0070  61 64 76 5f 66 69 6c 74  65 72 5f 70 6f 6c 69 63
>  |adv_filter_polic|
> 0080  79 3d 30 20 61 64 76 5f  69 74 76 6c 5f 6d 69 6e  |y=0
> adv_itvl_min|
> 0090  3d 30 20 61 64 76 5f 69  74 76 6c 5f 6d 61 78 3d  |=0
> adv_itvl_max=|
> 00a0  30 20 61 64 76 5f 64 61  74 61 5f 6c 65 6e 3d 30  |0
> adv_data_len=0|

It appears the log output is being sent over the newtmgr transport.  In
other words, text is getting logged to the console, and the newtmgr and
the cnosole are using the same uart (or a newtmgr transport isn't
enabled at all).  To solve this one, I would disable logging entirely
for the moment - Add the following to your target's syscfg.yml file:

syscfg.vals:
LOG_LEVEL: 255

And then send the above three newt commands.

Thanks,
Chris

Re: image upload error 2

[Christopher Collins]

Hi Then,

On Wed, Feb 22, 2017 at 02:11:36PM +0800, then yon wrote:
> Dear Support,
> 
> I'm followed the tutorial below:
> https://mynewt.apache.org/latest/os/modules/split/split/
> 
> But i'm stuck at image upload step; it gave me the following error:
[...]
> Error: Target error: 2

Error 2 indicates that there is no free slot for the new image.  In your
email, the "image list" output is a bit touch to read because it got
collapsed into a single line.  Here is my attempt to unravel it:

> $ newtmgr -c mynewt_serial image list
> Images:
> slot=0
> version: 1.0.0 
> bootable: true
> flags: active confirmed
> hash: 493c5e78e76a9c1b9e574ab38b7bbf0d342d928ffd0111400dbe8c2c952da68e
> 
> slot=1 
> version: 1.0.0
> bootable: false
> flags: active confirmed 
> hash: 
> 4d26a0d1596e12dab9059df3a13b4ffd54f96836cd9080d579c6afc167d58e81 
> 
> Split status: matching

Notice that both slots have the "active" and "confirmed" flags set.
This indicates that both slots are currently running.  You need to run
the image in loader-only mode in order to replace the app in slot 1.
This is described in the "Image Upgrade - Split" section of the page you
linked to (https://mynewt.apache.org/latest/os/modules/split/split/).
Specifically:


1. Disable split functionality; we need to deactivate the application
   image in slot 1 (newtmgr image test ).
2. Reboot device (newtmgr reset).
3. Make above change permanent (newtmgr image confirm).
4. Upload new loader to slot 1 (newtmgr image upload ).
5. Tell device to "test out" the new loader on next boot (newtmgr image
   test ).
6. Reboot device (newtmgr reset).
7. Make above change of loader permanent (newtmgr image confirm).
8. Upload new application to slot 1 (newtmgr image upload ).
9. Tell device to "test out" the new application on next boot (newtmgr
   image test ).
10. Reboot device (newtmgr reset).
11. Make above change of application permanent (newtmgr image confirm).

Steps 1 and 2 will leave the device in a state such that the app in slot
1 can be replaced.

Chris

Re: Config id package

[Christopher Collins]

Hi Jacob,

On Tue, Feb 21, 2017 at 10:34:53AM -0700, Jacob Rosenthal wrote:
> Im trying to build a DIS service with identification strings like bsp and
> app name from config/id package with conf_get_value
> 
> I think Im using the api correctly?
> 
> char *val;
> int rc;
> uint16_t uuid16;
> char tmp_buf[32 + 1]; ///hwid is only one that needs some tmp buffer
> 
> if(ctxt->op != BLE_GATT_ACCESS_OP_READ_CHR)
> {
> return BLE_ATT_ERR_UNLIKELY;
> }
> uuid16 = ble_uuid_u16(ctxt->chr->uuid);
> assert(uuid16 != 0);
> 
> switch (uuid16) {
> case BLE_SVC_DIS_CHR_SYS_ID_UUID16:
> val = conf_get_value("id/hwid", tmp_buf, sizeof(tmp_buf));
> console_printf("hwid %s\n", val);
> rc = os_mbuf_append(ctxt->om, val, strlen(val));
> return rc == 0 ? 0 : BLE_ATT_ERR_INSUFFICIENT_RES;
> 
> But I cant seem to get anything other than a empty string at runtime...
[...]

This is indeed a bummer.  The problem is that the conf_get_value()
function expects the setting name to be a mutable string.  String
literals (e.g., "id/hwid") are constant, and cause the setting lookup to
fail.  The reason the name needs to be mutable is that the lookup
routine calls strtok_r on the string, which replaces instances of '/'
with \0.

Anyway, the immediate fix is to put the setting name in a char array,
rather than pass a string literal to conf_get_value().  For example:

char conf_name[] = "id/hwid";

val = conf_get_value(conf_name, tmp_buf, sizeof(tmp_buf));

In my view, this aspect of the conf API isn't particularly obvious.  We
should either change the API such that it works with constant strings or
clearly document this requirement.

Chris

Re: os_msys_get_pkthdr always return null

[Christopher Collins]

Hi Then,

On Wed, Feb 15, 2017 at 10:58:57AM +0800, then yon wrote:
> After i tried on 3 NRF52_DK board, 1 of it worked!
> 
> This is really weird as the other 2 units only work if i disabled the 
> filesystem.
> 
> Do you have any clue on this?

I wonder if the flash is corrupt somehow.  There were some
backwards-compatibility-breaking changes to the boot loader and flash
maps between 0.9.0 and 1.0.0-b2.  I would try erasing the flash entirely
on one of the boards, then upload the boot loader and image again.

I wouldn't expect these symptoms from flash corruption, though...

Chris

Re: Can't build blinky app on macOS Sierra

[Christopher Collins]

Hi Denis,

On Wed, Feb 08, 2017 at 09:39:10PM -0800, Denis Magda wrote:
> Hello Mynewt community,
> 
> I tried to play with your product strictly following the getting started 
> guide [1] but can’t compile the default blinky app
> 
> Deniss-MBP:test dmagda$ newt build my_blinky_sim
> Building target targets/my_blinky_sim
> Assembling os_arch_stack_frame.s
> Error: os_arch_stack_frame.s:34:17: error: unexpected token in directive
> .globl CNAME(os_arch_frame_init)
> ^
> os_arch_stack_frame.s:39:26: error: unexpected token in argument list
> CNAME(os_arch_frame_init):
>  ^
> os_arch_stack_frame.s:84:19: error: unexpected token in memory operand
> callCNAME(sigsetjmp)/* sigsetjmp(sf->sf_jb, 0) */
>   ^
> os_arch_stack_frame.s:98:19: error: unexpected token in memory operand
> callCNAME(os_arch_task_start) /* os_arch_task_start(sf, rc) */

Hmm, that's odd.  I don't have any theories, but I'll look into it.
Could you please post the following:

* Contents of compiler/sim/compiler.yml
* Output of "gcc-6 -v" (or whatever your gcc binary is called)

Another option that could be helpful is to try building with the
"-ldebug" command line switch.  This will enable a lot of debug output,
including the actual command used to assemble that .s file.

Thanks,
Chris


> 
> 
> The dev environment is the following:
> * macOS Sierra
> * newt, gcc and gdb are natively installed
>   - newt version: Apache Newt (incubating) version: 1.0.0-dev
>  - gcc version: gcc version 6.3.0 (Homebrew GCC 6.3.0_1)
> * gcc-5 replaced with gcc-6 in compiler.yml according to this doc [2].  
>
> 
> Am I missing something or doing something wrong?
> 
> [1] https://mynewt.incubator.apache.org/os/get_started/project_create/
> [2] https://mynewt.incubator.apache.org/os/get_started/native_tools/
> 
> —
> Denis
> 

Re: [VOTE] Release Apache Mynewt 1.0.0-b2-incubating-rc1

[Christopher Collins]

Hi Szymon,

On Wed, Feb 08, 2017 at 03:55:44PM +0100, Szymon Janc wrote:
> blesplit application doesn't build on this tag:
> 
> Linking 
> /home/janc/devel/mynewt/bin/targets/blesplit/app/apps/blesplit/blesplit.elf
> Error: 
> /home/janc/devel/mynewt/bin/targets/blesplit/app/apps/blesplit/apps_blesplit.a(main.o):
> In function `main':
> /home/janc/devel/mynewt/repos/apache-mynewt-core/apps/blesplit/src/main.c:284:
> undefined reference to `g_dev_addr'
> collect2: error: ld returned 1 exit status

The blesplit app is only meant to be used as the second app in a split
image (yes, we should document this!).  The expectation is that bleprph
is used as the loader half.  For example, here is my blesplit-nrf52dk
target definition:

targets/blesplit-nrf52dk
app=apps/blesplit
bsp=hw/bsp/nrf52dk
build_profile=optimized
loader=apps/bleprph

I don't have any hardware available at the moment, but I will try
running this on a board in a little bit to see if I have the same issue.
This worked for me when I tried it a week or two ago, so yeah, plenty of
time for a bug to creep in :).

Personally, I don't think it is a worthwhile goal to ensure blesplit,
splitty, and other split apps can run as standalone applications.  They
are intended only for split images, and adding extra code to enable them
to run standalone seems counter to their purpose.

> 
> Enabling controller package makes it build fine but application
> asserts on start (probably due to controller and host initialization
> order).
> 
> So [-1] due to this.
> 
> -- 
> pozdrawiam
> Szymon K. Janc

Re: [VOTE] Release Apache Mynewt 1.0.0-b2-incubating-rc1

[Christopher Collins]

On Mon, Feb 06, 2017 at 05:32:55PM -0800, will sanfilippo wrote:
> [X] +1 Release this package
> [ ]  0 I don't feel strongly about it, but don't object
> [ ] -1 Do not release this package because…

+1 (binding)

Chris

Re: sysint() fails

[Christopher Collins]

Hi David,

Could your version of the newt tool be out of date?  Some backwards
compatibility breaking changes were made about two weeks ago.  If that
isn't the problem, could you grab a backtrace in gdb at the point of the
crash ("bt" or "where" in gdb)?

Thanks,
Chris


On Tue, Feb 07, 2017 at 09:43:19AM -0500, David G. Simmons wrote:
> Having some trouble this morning with the nrf52dk board.
> 
> 389   sysinit();
> (gdb) n
> 
> Program received signal SIGTRAP, Trace/breakpoint trap.
> __assert_func (file=file@entry=0x0, line=line@entry=0, func=func@entry=0x0, 
> e=e@entry=0x0) at 
> repos/apache-mynewt-core/kernel/os/src/arch/cortex_m4/os_fault.c:125
> 125  asm("bkpt");
> 
> I've updated both mynewt_nordic and apache-mynewt-core to the latest develop 
> branches, and
> 
> int
> main(int argc, char **argv)
> {
> int rc;
> 
> /* Initialize OS */
> sysinit();
> 
> ...
> 
> Fails at sysinit()
> 
> I've built a new bootloader (just in case). I thought maybe it was something 
> I was doing in my app, so I built and loaded core/apps/bleprph and
> 
> 259   sysinit();
> (gdb) n
> 
> Program received signal SIGTRAP, Trace/breakpoint trap.
> __assert_func (file=file@entry=0x0, line=line@entry=0, func=func@entry=0x0, 
> e=e@entry=0x0) at 
> repos/apache-mynewt-core/kernel/os/src/arch/cortex_m4/os_fault.c:125
> 125  asm("bkpt");
> 
> So it appears that something is broken for at least the nrf52dk dev board ...
> 
> cd repos/apache-mynewt-core/
> DSimmons-Pro:apache-mynewt-core dsimmons$ git status -v
> On branch develop
> Your branch is up-to-date with 'origin/develop'.
> cd ../mynewt_nordic/
> DSimmons-Pro:mynewt_nordic dsimmons$ git status -v
> On branch develop
> Your branch is up-to-date with 'origin/develop'.
> nothing to commit, working tree clean
> 
> dg
> --
> David G. Simmons
> (919) 534-5099
> Web  • Blog  • 
> Linkedin  • Twitter 
>  • GitHub 
> /** Message digitally signed for security and authenticity.
> * If you cannot read the PGP.sig attachment, please go to
>  * http://www.gnupg.com/  Secure your email!!!
>  * Public key available at keyserver.pgp.com 
> **/
> ♺ This email uses 100% recycled electrons. Don't blow it by printing!
> 
> There are only 2 hard things in computer science: Cache invalidation, naming 
> things, and off-by-one errors.
> 
> 

Re: BLE HCI support on NRF52DK

[Christopher Collins]

Hi Alan,

On Thu, Feb 02, 2017 at 06:16:42PM +, Alan Graves wrote:
> Thanks for rechecking. I know I'm not setup with RTS/CTS handshaking on the 
> custom board so that is probably going to be a problem. When I get a chance 
> I'll repeat my tests with a complete serial signal set on the nRF52832 DK 
> board. 
> 
> BTW Do you know if Linux assumes hardware flow control by default and if it 
> is possible to override that configuration somewhere? I've been caught before 
> with hardware flow defaults on PPP over serial on Linux... 

Yes, the btattach program assumes flow control and baud rate of 100
(recent versions allow you specify baud on the command line).

Flow control can be disabled with a simple code change to
tools/btattach.c:

*** btattach.c  2017-02-02 12:15:05.030918824 -0800
--- /home/admin/tmp/btattach.c  2017-02-02 12:14:59.951029001 -0800
***
*** 76,85 
--- 76,86 
cfmakeraw();
  
ti.c_cflag |= (speed | CLOCAL | CREAD);
  
/* Set flow control */
-   ti.c_cflag |= CRTSCTS;
+   ti.c_cflag &= ~CRTSCTS;
  
if (tcsetattr(fd, TCSANOW, ) < 0) {
perror("Failed to set serial port settings");
close(fd);

Then run make from the bluez source directory.  The new btattach binary
will be named tools/btattach.

You also need to disable flow control in your Mynewt target:

newt target set nina_hci 
syscfg=BLE_HCI_UART_FLOW_CTRL=HAL_UART_FLOW_CTL_NONE

And rebuild / reload the firmware onto your board.

Chris

Re: NimBLE host advertising data API

[Christopher Collins]

Thanks Andrzej.  That makes sense.

Chris

On Fri, Jan 27, 2017 at 04:57:48PM +0100, Andrzej Kaczmarek wrote:
> Hi Chris,
> 
> I created a pull request which also updates API to raw data for the
> opposite direction, i.e. from host to app. Application can still parse this
> to ble_hs_adv_fields since helper is now public, but has to do this
> explicitly. The advantage is that if application prefers to work on raw
> data (I added simple iterator to make this easier) some extra code and
> static variables are removed saving some flash.
> 
> There are also some extra fixes includes for advertising API I had pending
> in my tree.
> 
> Link: https://github.com/apache/incubator-mynewt-core/pull/169
> 
> BR,
> Andrzej
> 
> 
> 
> On Thu, Jan 26, 2017 at 4:16 AM, Christopher Collins <ccoll...@apache.org>
> wrote:
> 
> > Heads up: I have pushed this API change.
> >
> > The new functions are:
> > int ble_gap_adv_set_data(const uint8_t *data, int data_len);
> > int ble_gap_adv_rsp_set_data(const uint8_t *data, int data_len);
> >
> > /* Converts a high-level set of fields to a byte buffer. */
> > int ble_hs_adv_set_fields(const struct ble_hs_adv_fields *adv_fields,
> >   uint8_t *dst, uint8_t *dst_len,
> >   uint8_t max_len):
> >
> > The old functions are still around because they are convenient
> > (ble_gap_adv_set_data() and ble_gap_adv_rsp_set_data()); they won't get
> > included in the build if your app does not call them.
> >
> > If you use ble_hs_adv_set_fields() or the old functions, please be aware
> > of one more API change: you need to explicitly specify the flags value
> > that you want to include in advertisements.  Before this change, you
> > could specify a value of 0, and the flags field would get
> > auto-calculated by the host.
> >
> > E.g.,
> >
> > OLD API:
> >
> > fields.flags_is_present = 1;
> > fields.flags = 0;
> >
> > NEW API:
> >
> > fields.flags = BLE_HS_ADV_F_DISC_GEN |
> >BLE_HS_ADV_F_BREDR_UNSUP;
> >
> > Thanks,
> > Chris
> >
> > On Tue, Jan 24, 2017 at 11:45:33AM -0800, Christopher Collins wrote:
> > > Hello all,
> > >
> > > I've mentioned this before - I really don't care for the advertising
> > > data API that we ended up with:
> > > http://mynewt.apache.org/latest/network/ble/ble_hs/ble_
> > gap/functions/ble_gap_adv_set_fields/
> > >
> > > I think we should change this API now before the 1.0 release.  I was
> > > wondering what others think.
> > >
> > > The current API is high-level and is relatively easy to use, but
> > > requires a lot of code space and RAM.  I think a function which just
> > > takes a raw byte buffer (or mbuf) would be much better.  Then, there
> > > could be a helper function which converts an instance of `struct
> > > ble_hs_adv_fields` to a raw byte buffer.
> > >
> > > A simple peripheral that always advertises the same data shouldn't be
> > > burdened with the ble_hs_adv_fields API.
> > >
> > > There is actually a rationale as to why the API is the way it is today.
> > > There are a few fields in the advertisement data that the host can be
> > > configured to fill in automatically:
> > > * Flags (contains advertising type).
> > > * TX Power Level
> > >
> > > I figured it would be safer if these values got calculated when
> > > advertising is initiated.  This is impractical if the host were handed a
> > > byte buffer rather than a series of fields.
> > >
> > > Under the new proposal, the application would need to specify the
> > > correct advertising type when building the byte buffer, and the tx power
> > > level would be queried before the advertising procedure is actually
> > > started.  I don't think this will be a problem in practice, and I think
> > > the benefits in code size and RAM usage outweigh the API burden.
> > >
> > > All thoughts welcome.
> > >
> > > Thanks,
> > > Chris
> >

Re: newNewt

[Christopher Collins]

Sorry, there's something else I forgot to mention.  You'll need to put
the boot loader on your board as well.  You can do this before or after
uploading blinky.

newt target create boot-frdm-k64f &&
newt target set boot-frdm-k64f app=@apache-mynewt-core/apps/boot\
   bsp=@apache-mynewt-core/hw/bsp/frdm-k64f \
   build_profile=optimized

Then build and upload the boot loader to your board:

newt build boot-frdm-k64f
newt load boot-frdm-k64f

Chris


On Thu, Jan 26, 2017 at 09:14:02AM -0800, Christopher Collins wrote:
> To start with, I would create a blinky-frdm-k64f target:
> 
> newt target copy my_blinky_sim blinky-frdm-k64f
> 
> Then configure your new target to use the frdm-k64f BSP:
> 
> newt target set blinky-frdm-k64f bsp=@apache-mynewt-core/hw/bsp/frdm-k64f
> 
> Plug your board in and attach a debugger if necessary, and try running
> blinky on your board:
> 
> newt run blinky-frdm-k64f
> 
> If everything works, a gdb window will come up.  Type c , and
> check if your board's LED is blinking.
> 
> After you have blinky working on your board, you might want to try one
> of these other sample apps:
> slinky: Includes shell over UART and newtmgr over shell.
> bleprph: Includes BLE stack and newtmgr over BLE.
> 
> Thanks,
> Chris
> 
> > 
> > Alternatively, I could familiarize myself work through the Olimex-E407
> > 
> > My  environment so far has been IDE Freescale Kinetis Design
> > Studio/Eclipse building nuttx OS, and with integration to Multilink JTAG
> > for blowing the flash.
> > 
> > There was a new feature request for an Eclipse plugin for myNewt, but it 
> > was marked as dup, and I haven't seen any other mention of Eclipse IDE so
> > far.
> > 
> > Any recommendation on getting started with FRDM-K64F?   or should I
> > start with Olimex
> > 
> > thanks
> > 
> > 
> > -- 
> > Neil Hancock
> > 
> > 
> > 

Re: newNewt

[Christopher Collins]

On Thu, Jan 26, 2017 at 09:14:02AM -0800, Christopher Collins wrote:
> Plug your board in and attach a debugger if necessary, and try running
> blinky on your board:
> 
> newt run blinky-frdm-k64f

That should be:

newt run blinky-frdm-k64f 0

(append a zero!)

Chris

Re: Newtmgr over BLE

[Christopher Collins]

Hi Kevin,

On Wed, Jan 25, 2017 at 02:18:01AM +0100, Kevin Townsend wrote:
> I'm not sure what the implications would be here on the iOS app we wrote 
> that is based on the newtmgr protocol and makes use of the current BLE 
> service and characteristics, although we could rewrite it if necessary. 
> We use the currently defined newtmgr protocol and GATT services/chars to 
> get task info and statistic and stream them live onto the mobile device, 
> which I think provides a unique window into your device in a 
> non-invasive way.

The newtmgr protocol wouldn't be affected.  This would just be a way to
get the newtmgr tool to speak BLE.

> I'm certainly OK with BLE being removed from the newtmgr tool and having 
> that supported by a dedicated tool or service based on UART/HCI, but 
> just to be sure are there any implications here for the newtmgr protocol 
> running on a mynewt app?  It sounds like we should still be OK with the 
> app, though, and this is limited to the newtmgr tool itself not the 
> newtmgr protocol implementation or the BLE transport in the Mynewt app.
> 
> One advantage of taking the HCI approach talking to a Mynewt target over 
> UART is that you can add support to any OS, not just systems that 
> support BLE, and not having to depend on a specific OS or BlueZ version. 
> You need to do some work up front and it would require another nRF5x 
> board, but it solves more problems than it creates in my opinion.
> 
> Alternatively, you could use the simulator and try to add some hooks 
> into CoreBluetooth and BlueZ (so no additional HW required), but that's 
> ultimately frustrating and a significant maintenance burden. UART/HCI 
> seems like less work both short and long term, and the requirements for 
> an additional nRF5x board seems relatively trivial.

All good points.  I really think BLE on Linux vs. MacOS is so different
that it would be less of a headache to make our own.

> Removing the dependency on the current GO lib and moving to HCI/UART and 
> a dedicated Mynewt app is a +1 for me and worth doing before 1.0, even 
> if it's a major change ... with the caveat that I hope this won't break 
> anything with the newtmgr protocol that will cause us to have to rewrite 
> the newtmgr tool we wrote and want to release around the same time as 
> 1.0.0 final. :)

Thanks,
Chris

> On 24/01/17 23:49, Christopher Collins wrote:
> > Hello all,
> >
> > Recently, I have mentioned some planned BLE-related changes to the
> > newtmgr tool.  I wanted to share some of what I was thinking.  Please
> > feel free to comment and criticize as needed.
> >
> > * All BLE code gets removed from the newtmgr tool.  The gatt library is
> >also removed.
> >
> > * A separate tool, blehostd, runs in its own process.  This tool:
> >  * Is a Mynewt app running in the Mynewt simulator.
> >  * Contains the NimBLE host and UART BLE transport.
> >  * Caches information about connected peers (who is connected and how
> >their services, characteristics, and descriptors map to attribute
> >handles).
> >  * Communicates with other processes via JSON over a streaming UNIX 
> > domain
> >socket.
> >  * Exposes the host API via JSON-encoded requests, responses, and
> >events.
> >
> > * Controller is accessed via UART device (/dev/cu.<...>).  This could be
> >a Mynewt device running the blehci app, or any other controller.
> >
> > The blehostd app solves a few problems for us:
> >  * Fixes the bugginess and lack of cross platform support in
> >newtmgr (caused by the gatt library).
> >  * Exposes a generic interface into the NimBLE host for software
> >other than newtmgr.
> >
> > Here is how I envision newtmgr using BLE:
> >
> > 1. If an instance of blehostd isn't already running for this
> > controller [*]:
> >  a. Open and bind to a socket.
> >  b. Start an instance of blehostd, passing it the socket filename
> > and the controller's /dev filename.
> >
> > 2. Newtmgr tells blehostd to connect to the destination device.
> >  If the device is already connected: blehostd immediately responds,
> >  indicating the connection handle of the device.
> >  Else: The response indicates that the connection attempt is in
> >  progress.  Eventually, blehostd sends an event indicating
> >  success and a connection handle, or failure.
> >
> > 3. Newtmgr asks blehostd if the peer's newtmgr characteristic has been
> > discovered yet.
> >  Yes: blehostd's response indicates the characteristic's
>

Re: NimBLE host advertising data API

[Christopher Collins]

On Tue, Jan 24, 2017 at 12:40:04PM -0800, will sanfilippo wrote:
> I am not sure I have any intelligent comments on this, but that has never 
> stopped me from commenting in the past, so…

No worries.  Thanks for the feedback!

> 
> I think a byte buffer interface is fine as long as you have helper functions 
> to create that buffer. Having folks have to figure out how to create an 
> advertisement without any helper functions would be a bad idea (imho).
> 
> I am not sure I completely understand your example re:Tx Power Level. Would 
> this field still get added by the host or would there be a helper function 
> that a developer could call to add the Tx Power Level field to the 
> advertisement?

The host wouldn't modify the advertising data at all.  If the
application wants to advertise the tx power level, it would need to
arrange for it to be written to the byte buffer.  If using the helper
function, the application would write the correct value to the
tx_pwr_lvl field in the ble_hs_adv_fields struct before converting the
struct to a byte array.  The application would either "just know" the
correct value, or it would query the host prior to building the
advertising data buffer.

Chris

Newtmgr over BLE

[Christopher Collins]

Hello all,

Recently, I have mentioned some planned BLE-related changes to the
newtmgr tool.  I wanted to share some of what I was thinking.  Please
feel free to comment and criticize as needed.

* All BLE code gets removed from the newtmgr tool.  The gatt library is
  also removed.

* A separate tool, blehostd, runs in its own process.  This tool:
* Is a Mynewt app running in the Mynewt simulator.
* Contains the NimBLE host and UART BLE transport. 
* Caches information about connected peers (who is connected and how
  their services, characteristics, and descriptors map to attribute
  handles).
* Communicates with other processes via JSON over a streaming UNIX domain
  socket.
* Exposes the host API via JSON-encoded requests, responses, and
  events.

* Controller is accessed via UART device (/dev/cu.<...>).  This could be
  a Mynewt device running the blehci app, or any other controller.

The blehostd app solves a few problems for us:
* Fixes the bugginess and lack of cross platform support in
  newtmgr (caused by the gatt library).
* Exposes a generic interface into the NimBLE host for software
  other than newtmgr.

Here is how I envision newtmgr using BLE:

1. If an instance of blehostd isn't already running for this
   controller [*]:
a. Open and bind to a socket.
b. Start an instance of blehostd, passing it the socket filename
   and the controller's /dev filename.

2. Newtmgr tells blehostd to connect to the destination device.
If the device is already connected: blehostd immediately responds,
indicating the connection handle of the device.
Else: The response indicates that the connection attempt is in
progress.  Eventually, blehostd sends an event indicating
success and a connection handle, or failure.

3. Newtmgr asks blehostd if the peer's newtmgr characteristic has been
   discovered yet.
Yes: blehostd's response indicates the characteristic's
attribute handle.
No: newtmgr tells blehostd to discover the newtmgr
characteristic.  After discovery completes, newtmgr repeats
this step so that it knows the characteristic's value handle.

4. Newtmgr builds the CBOR newtmgr request, as always.

5. Newtmgr encodes the CBOR newtmgr request as a bluetooth ATT write
   command, and tells blehostd to write it to the peer's newtmgr
   characteristic.

6. blehostd immediately responds, indicating success or failure.

7. If successful, blehostd sends an event containing to the peer's
   notification (corresponding to its newtmgr response).

[*] This might be a bit tricky to get right.  The goal is to not have to
connect and perform service discovery for each newtmgr command.
Ideally, blehostd would stick around until some idle timeout is
exceeded.  Then, newtmgr could just reuse the existing socket and
blehostd instance to immediately send a follow-up command.  This is more
of an optimization, so it can probably come later.

All comments welcome.

Thanks,
Chris

Re: push more often (was: [01/50] incubator-mynewt-core git commit: ...)

[Christopher Collins]

Those commits were made to a different branch:

> >>> Repository: incubator-mynewt-core
> >>> Updated Branches:
> >>>  refs/heads/sensors_branch 6247b5afa -> 2681044e8

That mass of commits was just a big merge from develop to
sensors_branch.  Sterling was just bringing sensors_branch up to date
with develop.

I've also been confused by git commit emails sometimes.  You have to
double check the branch name!

Chris

On Tue, Jan 24, 2017 at 04:02:40PM +, Wayne Keenan wrote:
> 
> 
> > On 24 Jan 2017, at 15:56, Peter Saint-Andre - Filament  
> > wrote:
> > 
> > Things seem pretty transparent around here to me!
> > 
> 
> I'd say so, and I'd trust Sterling's commits, but,  'Review Harder', if you 
> like :)
> 
> >> On 1/24/17 12:41 AM, Greg Stein wrote:
> >> commit 1 of 50 ??
> >> 
> >> This says to me: push more often. How can the mynewt community review your
> >> work, if you never push it?
> >> 
> >>> On Mon, Jan 23, 2017 at 9:02 PM,  wrote:
> >>> 
> >>> Repository: incubator-mynewt-core
> >>> Updated Branches:
> >>>  refs/heads/sensors_branch 6247b5afa -> 2681044e8
> >>> 
> >>> 
> >>> nimble/sm: Use TinyCrypt for AES
> >>> 
> >>> TinyCrypt is smaller than mbedTLS and is already used for ECDH.
> >>> Using TC for all crypto in SM results in following code size reductions
> >>> for bletiny application:
> >>> 
> >>> Legacy Pairing only from
> 250 277 *fill*
>   11160   0 crypto_mbedtls.a
>   485813410 net_nimble_host.a
>  1449922784   15788  163564   27eec apps/bletiny/bletiny.elf
> >>> 
> >>> to
> >>> < 252 277 *fill*
> >>> <1112   0 crypto_tinycrypt.a
> >>> <   485633130 net_nimble_host.a
> >>> <  1349282784   15508  153220   25684 app/apps/bletiny/bletiny.elf
> >>> 
> >>> Legacy + LE SC from
> 264 276 *fill*
>   11160   0 crypto_mbedtls.a
>   518813627 net_nimble_host.a
>  1522722980   16004  171256   29cf8 app/apps/bletiny/bletiny.elf
> >>> 
> >>> to
> >>> < 254 276 *fill*
> >>> <   518633347 net_nimble_host.a
> >>> <  1410842980   15724  159788   2702c app/apps/bletiny/bletiny.elf
> >>> 
> >>> 
> >>> Project: http://git-wip-us.apache.org/repos/asf/incubator-mynewt-core/repo
> >>> Commit: http://git-wip-us.apache.org/repos/asf/incubator-mynewt-
> >>> core/commit/2785cad5
> >>> Tree: http://git-wip-us.apache.org/repos/asf/incubator-mynewt-
> >>> core/tree/2785cad5
> >>> Diff: http://git-wip-us.apache.org/repos/asf/incubator-mynewt-
> >>> core/diff/2785cad5
> >>> 
> >>> Branch: refs/heads/sensors_branch
> >>> Commit: 2785cad50147160d21bef9aef143199f294ed093
> >>> Parents: a46fdfe
> >>> Author: Szymon Janc 
> >>> Authored: Wed Jan 18 14:24:44 2017 +0100
> >>> Committer: Szymon Janc 
> >>> Committed: Wed Jan 18 14:54:44 2017 +0100
> >>> 
> >>> --
> >>> net/nimble/host/pkg.yml  |  2 +-
> >>> net/nimble/host/src/ble_sm_alg.c | 21 +++--
> >>> 2 files changed, 8 insertions(+), 15 deletions(-)
> >>> --
> >>> 
> >>> 
> >>> http://git-wip-us.apache.org/repos/asf/incubator-mynewt-
> >>> core/blob/2785cad5/net/nimble/host/pkg.yml
> >>> --
> >>> diff --git a/net/nimble/host/pkg.yml b/net/nimble/host/pkg.yml
> >>> index f7539a4..d025934 100644
> >>> --- a/net/nimble/host/pkg.yml
> >>> +++ b/net/nimble/host/pkg.yml
> >>> @@ -31,7 +31,7 @@ pkg.deps:
> >>> - util/mem
> >>> 
> >>> pkg.deps.BLE_SM_LEGACY:
> >>> -- crypto/mbedtls
> >>> +- crypto/tinycrypt
> >>> 
> >>> pkg.deps.BLE_SM_SC:
> >>> - crypto/tinycrypt
> >>> 
> >>> http://git-wip-us.apache.org/repos/asf/incubator-mynewt-
> >>> core/blob/2785cad5/net/nimble/host/src/ble_sm_alg.c
> >>> --
> >>> diff --git a/net/nimble/host/src/ble_sm_alg.c
> >>> b/net/nimble/host/src/ble_sm_alg.c
> >>> index 8a5365d..f8208b4 100644
> >>> --- a/net/nimble/host/src/ble_sm_alg.c
> >>> +++ b/net/nimble/host/src/ble_sm_alg.c
> >>> @@ -28,20 +28,15 @@
> >>> #include "nimble/ble.h"
> >>> #include "nimble/nimble_opt.h"
> >>> #include "ble_hs_priv.h"
> >>> -#include "mbedtls/aes.h"
> >>> -
> >>> -#if MYNEWT_VAL(BLE_SM_SC)
> >>> -
> >>> #include "tinycrypt/aes.h"
> >>> #include "tinycrypt/constants.h"
> >>> #include "tinycrypt/utils.h"
> >>> +
> >>> +#if MYNEWT_VAL(BLE_SM_SC)
> >>> #include "tinycrypt/cmac_mode.h"
> >>> #include "tinycrypt/ecc_dh.h"
> >>> -
> >>> #endif
> >>> 
> >>> -static mbedtls_aes_context ble_sm_alg_ctxt;
> >>> -
> >>> static void
> >>> ble_sm_alg_xor_128(uint8_t *p, uint8_t *q, uint8_t *r)
> >>> {
> >>> @@ -55,22 +50,20 @@ ble_sm_alg_xor_128(uint8_t *p, uint8_t *q, uint8_t *r)
> >>> static int
> >>> ble_sm_alg_encrypt(uint8_t *key, uint8_t *plaintext, uint8_t 

Re: bleuart example causes crash

[Christopher Collins]

Hi Chew,

It looks like the problem is a broken task handler.  In bleuart's
main.c, the bleuart_task_handler() function should call os_eventq_run()
in an infinite loop, but it only calls it once.

Marko just fixed this bug in develop with his default task changes.

Chris

On Fri, Jan 20, 2017 at 06:00:45PM +, Lm Chew wrote:
> [https://tr.cloudmagic.com/h/v6/emailtag/tag/1484935225/e07327a00ce48c6feaf5c6318fd5666d/98148963e301c00d168ac75868cda51d/89bf195313601b8fc94e962f71ed9d07/7d73f5719844943a877cfefbca240ecc/newton.gif]
> 
> Hi Chris,
> 
> Thanks.
> 
> Yeah, it is good to have for sending debug messages and receiving command 
> wirelessly for device that doesn't have physical UART interface built into 
> the design.
> 
> Best Regards,
> Chew
> 
> 
> On Sat, Jan 21, 2017 at 12:34am, Christopher Collins 
> <ccoll...@apache.org<mailto:ccoll...@apache.org>> wrote:
> 
> On Fri, Jan 20, 2017 at 09:41:24AM +, Lm Chew wrote:
> > Hi,
> >
> >
> > Does any one have problem running the bleuart example?
> >
> > When I run it, it crashes in the end with Unhandled interrupt error. (I am 
> > using bmd300eval)
> > I got the following debug output:
> 
> It has been quite a while since I have run bleuart.  I'll take a look at
> it today.  In the meantime, are you sure you need the bleuart
> functionality?  Just to be cler, this app implements a somewhat scaled
> down version of the Nordic BLE serial port emulation service.
> 
> Chris

Re: Program board without J-Link/OpenOCD.

[Christopher Collins]

Hi Marcos,

I don't know if I can solve this problem, but I can at least answer a
few of your questions.  Hopefully someone more knowledgeable can fill in
the gaps.

On Mon, Jan 23, 2017 at 02:34:11PM -0200, Marcos Scheeren wrote:
[...]
> To my understanding, the newt create-image command outputs an .img with the
> bootloader sector, the sectors headers, and the two application sectors,
> needed for the bootloader to properly locate, identify and load the
> applications.

The .img file only contains:
* image header
* image binary

The contents of a .img file occupy a single image slot.  The boot loader
slot and the second image slot are untouched when you upload an image.

A .img file, as opposed to a .bin file, is only useful if you are using
the boot loader.  The boot loader knows how to read the image header and
determine where the image executable actually begins.  If you aren't
using a boot loader, then you need to upload a .bin file to address 0 so
that the hardware jumps straight into the image on boot.  This process
is minimally described here:
https://mynewt.apache.org/latest/os/modules/split/split/

> And my "low-level bootloader knowledge" ends here :( So the questions are:
> 
> - Are there any ways to output a single .elf or .bin file already with the
> sectors headers (maybe convert the .img file) so I can load it using the
> GDB extended-remote mode? Or that would require further work, like changing
> the newt golang sources?

That sounds like a manufacturing image.  A manufacturing image contains
the entire contents of a device's flash; it is intended to be built into
every unit during the early stages of manufacturing.  In particular, a
manufacturing image contains:
* boot loader
* image in slot 0
* (optional) image in slot 1 or split image
* (optional) user data

There is some basic documentation for creating mfg images here:
http://mynewt.apache.org/latest/newt/command_list/newt_mfg/

This process should definitely work, as we use it frequently.  However,
the documentation may be lacking, so please don't hestitate to follow up
with questions.

> Thanks.
> Other than that, really appreciate the architecture of the OS!

Chris

Re: MBUF sizing for the bluetooth stack

[Christopher Collins]

On Sun, Jan 22, 2017 at 09:16:59AM -0800, Christopher Collins wrote:
> There are some optimizations that could be done here.  In most cases,
> the host is interested in the full attribute value.  The host should
> probably allocate GATT read mbufs using ble_hs_mbuf_att_pkt() in these
> cases so that it can just send the mbuf after the application has filled
> it.

Btw, I went ahead and implemented / pushed this optimization.

Chris

Re: MBUF sizing for the bluetooth stack

[Christopher Collins]

On Fri, Jan 20, 2017 at 02:27:41PM -0800, Simon Ratner wrote:
> I am talking about GATT reads, where the mbuf is pre-allocated for the app
> to fill in:
> 
> - static int
> - gatt_svr_chr_access_gap(
> -   uint16_t conn_handle,
> -   uint16_t attr_handle,
> -   struct ble_gatt_access_ctxt *ctxt, void *arg)
> - {
> -   ...
> -   os_mbuf_append(ctxt->om, data, len);
> -   ...
> - }
> -
> - Similarly, GATT read ops triggered by ble_gatt_chr_updated for sending
> out notifications.

These mbufs never actually get sent to the controller, so they don't
need the extra leading space.  After the application fills one of these
mbufs with a characteristic's value, the host copies the relevant data
into a properly allocated mbuf and sends the copy.

This may seem wasteful (in fact, that's what I'm thinking as I write
this!), but there is at least some logic here.  In the case of a GATT
read, the peer may read from different offsets of the characteristic.
In an effort to simplify the API, I decided the requested offset should
be hidden from the application.  Instead, the application always
provides the full characteristic value, and the host copies the
requested portion out of the user-filled mbuf and into the ACL data
packet.

There are some optimizations that could be done here.  In most cases,
the host is interested in the full attribute value.  The host should
probably allocate GATT read mbufs using ble_hs_mbuf_att_pkt() in these
cases so that it can just send the mbuf after the application has filled
it.

Alternatively, the host could specify the offset and length being read,
and require the application to provide only the requested range of
bytes.  This would be a little messier, but would be the most mbuf
efficient.

I'm thinking we should do the first optimization immediately.  I don't
believe it would be much work, and I don't see any downsides.  I'll take
a closer look at it today.

Chris

Re: Firmware update with MyNewt

[Christopher Collins]

Hi Then,

On Sun, Jan 22, 2017 at 08:39:45AM +0800, then yon wrote:
> Dear Jacob,
> 
> Thanks for your verification, so that means i need to use newtmgr for 
> ota update.
> 
> I tried on newtmgr but i stuck when i tried to do something with the 
> remote endpoint.
> 
> Here what i did:
> 
> 
> Then it stuck there, it same for any other command that required 
> interaction with remote endpoint.
> 
> My remote endpoint is nrf52dk board with bootloader and a working app 
> loaded.
> 
> Another noob question here, how do i add a newtmge ota update conn? Is 
> that add a ble conn?

I recommend trying the "slinky" app.  This app is configured to support
newtmgr and image management, so you can use it to verify that your
serial setup is correct.

To support newtmgr in your app, you will need to:

1. Make sure the following packages get pulled in to the build:

- mgmt/imgmgr
- mgmt/newtmgr
- mgmt/newtmgr/transport/nmgr_shell
- sys/console/full
- sys/shell

The easiest way to do this is to add these entries to your app's or
target's pkg.yml file, in the "pkg.deps" section.

2. Enable the SHELL_TASK syscfg setting.  You can do this by adding the
following lines to your app's or target's syscfg.yml file:

syscfg.vals:
# Enable the shell task.
SHELL_TASK: 1

If "syscfg.vals" is already present in the file, then you just need to
add the "SHELL_TASK: 1" line under it.

Chris

Re: MBUF sizing for the bluetooth stack

[Christopher Collins]

On Fri, Jan 20, 2017 at 01:21:20PM -0800, Simon Ratner wrote:
> On Fri, Jan 20, 2017 at 9:01 AM, Christopher Collins <ccoll...@apache.org>
> wrote:
> > On Thu, Jan 19, 2017 at 11:57:01PM -0800, Simon Ratner wrote:
> > > When allocating mbufs for the payload, is there something I should do to
> > > reserve enough leading space for the ACL header to make sure host doesn't
> > > need to re-allocate it?
> >
> > Yes - you should use ble_hs_mbuf_att_pkt() to allocate all your mbufs.
> > This function ensures the resulting mbuf has sufficient leading space
> > for:
> > * ACL data header
> > * Basic L2CAP header
> > * Four bytes for an ATT command
> >
> 
> And I assume the mbuf passed into gatt read handlers (ctxt->om) has been
> allocated that way? I only need to worry about the mbufs I allocate myself,
> such as for notify payload.

The mbufs handed up from the host are typically allocated by something
else (e.g., for the combined host-controller, it is the controller which
allocates them).  If you pass these mbufs back to the host, there is a
chance the host will need to allocate an additional buffer while
prepending to the chain.  The host stripped the headers from the front
of the mbuf before handing it to the application, so there will be
sufficient leading space to add them back.  However, the problem is the
variable length ATT command.  Your outgoing command may require more
"ATT bytes" than the one you received.

Do you often pass these mbufs back to the host?  I thought that would be
an uncommon use case.

Chris

Re: [RFC] Reducing size of BLE Security Manager

[Christopher Collins]

On Fri, Jan 20, 2017 at 09:45:07AM -0800, will sanfilippo wrote:
> I was referring to C code that accesses a packed structure, not necessarily 
> the construction part of it. For example: (and in this example I am assuming 
> the processor can access bytes anywhere, 16-bit values on 16-bit boundaries 
> and 32-bit values on 32-bit boundaries).
> 
> struct my_struct
> {
>   uint8_t e8;
>   uint16_t e16;
>   uint32_t e32;
> } __packed__  /* I know this syntax is wrong, just an example */
> struct my_struct my_data
> 
> In your C code when you do this: my_data.e32 = 50, what is the
> compiler going to do? If the structure is not packed, it knows it can
> use an instruction that accesses words. If the structure is packed,
> well, I guess it is up to the compiler what to do. In the past, I have
> seen compilers that add code or call functions that will check the
> alignment of e32. If e32 happens to reside on a 4-byte boundary it
> will use a word instruction; if it happens to reside on a byte
> boundary it needs to access the bytes individually to put them in a
> register for use.

I'm not really adding anything here, but here is something I realized
recently.  When you tell gcc to pack a struct, it has two effects:

1. Eliminates padding.
2. Assumes instances of the struct are not properly aligned.

For MCUs which don't support unaligned accesses, the second effect may
carry some hidden costs.  Even if the struct is defined such that it
wouldn't contain any padding, and even if all instances of the struct
are properly aligned, adding the __packed__ attribute will result in an
increase in code size.  The increase occurs because gcc can no longer
assume that the struct or any of its members are aligned.

Chris

Re: MBUF sizing for the bluetooth stack

[Christopher Collins]

On Thu, Jan 19, 2017 at 11:57:01PM -0800, Simon Ratner wrote:
> Thanks Chris,
> 
> It appears to me that there is questionable benefit to having mbufs sized
> larger than the largest L2CAP fragment size (plus overhead), i.e. the 80
> bytes that Will mentioned. Is that a reasonable statement, or am I missing
> something?
> 
> For incoming data, you always waste memory with larger mbufs, and for
> outgoing data host will take longer to free the memory (since you can't
> free the payload mbuf until the last fragment, as opposed to freeing
> smaller mbufs as you go), and you don't save on the number of copies in the
> host. You will save something on mbuf allocations and mbuf header overhead
> in the app as you are generating the payload, though.

I agree with your analysis.  This is just for BLE data, of course.  If
you use msys for other things (e.g., an alternative to malloc for
internal use), that obviously has an affect on the optimum msys buffer
size.

> When allocating mbufs for the payload, is there something I should do to
> reserve enough leading space for the ACL header to make sure host doesn't
> need to re-allocate it?

Yes - you should use ble_hs_mbuf_att_pkt() to allocate all your mbufs.
This function ensures the resulting mbuf has sufficient leading space
for:
* ACL data header
* Basic L2CAP header
* Four bytes for an ATT command

The last item (four bytes) is imprecise because different ATT commands
have different payload sizes, and this function doesn't know which
command you'll be sending.  Four bytes is the most the host would never
need to prepend to application attribute data.

We should expose some more functions from
net/nimble/host/src/ble_hs_mbuf.c that give the user more control over
the amount of leading space.  This would be useful for when the
application wants to send an ATT command that doesn't need the full four
bytes.

Just to be clear - if you allocate an mbuf that lacks sufficient leading
space (e.g., via os_msys_get_pkthdr()) and pass it to the host, the host
won't reallocate and copy the entire chain; it will just allocate a
single buffer and prepend it to the chain.  This is still wasteful, but
it can be completely avoided by using the ble_hs_mbuf_att_pkt()
function.

> Also, at least in theory, it sounds like you could size mbufs to match the
> fragment exactly -- or pre-fragment the mbuf chain as you are generating
> the payload -- and have zero copies in the host. Could be useful in a
> low-memory situation, if the host was smart enough to take advantage of
> that?

The host isn't smart enough :).  The complication arises from the
"os_mbuf_pkthdr" that the leading buffer contains.  The presence of this
header causes the first buffer to have less capacity than subsequent
buffers.  The fragmentation procedure never frees the chain's leading
buffer.  The assumption is that the data is packed in the mbuf chain,
and that the second buffer doesn't have sufficient leading space to
contain the pkthdr.

A smarter procedure might check how much unused space the second buffer
contains.  If there is sufficient room for the pkthdr, the procedure
would move the data to make room for the pkthdr at the front of the
buffer, then copy the pkthdr into the buffer.  After this, the leading
buffer could be freed.  This might be worth looking into.

Thanks,
Chris

Re: Tutorial problem with the Docker Container

[Christopher Collins]

On Fri, Jan 20, 2017 at 11:55:59PM +0800, Li-Chun Ko wrote:
> As an update, the Docker Container works well with the "./newt test all"
> commend.

Great to hear.  Thanks for following up!

Chris

Re: bleuart example causes crash

[Christopher Collins]

On Fri, Jan 20, 2017 at 09:41:24AM +, Lm Chew wrote:
> Hi,
> 
> 
> Does any one have problem running the bleuart example?
> 
> When I run it, it crashes in the end with Unhandled interrupt error. (I am 
> using bmd300eval)
> I got the following debug output:

It has been quite a while since I have run bleuart.  I'll take a look at
it today.  In the meantime, are you sure you need the bleuart
functionality?  Just to be cler, this app implements a somewhat scaled
down version of the Nordic BLE serial port emulation service.

Chris

Re: [RFC] Reducing size of BLE Security Manager

[Christopher Collins]

On Fri, Jan 20, 2017 at 10:21:16AM +0100, Szymon Janc wrote:
> Code is available at [1]. I'm not making it a pull request yet since
> I'd like to get some feedback about this approach from others. Also I
> still need to get tests passing since SM keys related tests fail now
> (and I'm not yet sure why).   I tested this with Android phone and
> both legacy and LE SC seems to work just fine.

(you may already know all this...)

By the way, you can run the unit tests using:

newt run net/nimble/host/test

When you use the "run" command, the tests get run in gdb, and the first
test failure causes gdb to halt.  This is useful for debugging test
failures.

Some of the host unit tests are a bit... interesting to debug :).  I can
take a look at the failure if you like.

Chris

Re: [RFC] Reducing size of BLE Security Manager

[Christopher Collins]

Hi Szymon,

On Fri, Jan 20, 2017 at 10:21:16AM +0100, Szymon Janc wrote:
> Hi,
> 
> I was recently looking on how we could reduce size of SM code.
> So my proposal is to change the way PDUs are parsed and constructed.
> 
> Instead of having ble_sm_foo_parse(), ble_sm_foo_write() and ble_sm_foo_tx()
> for parsing and constructing PDU byte by byte we could use packed structures
> for describing PDU and let compiler figure out details related to
> unaligned access.
[...]

I think that's a great idea.  The ATT code does something similar,
though there is probably more work to be done there.  In my opinion,
using packed structs for parsing and encoding doesn't just reduce code
size, it also simplifies the code.

Chris

Re: Tutorial problem with the Docker Container

[Christopher Collins]

Hi Li-Chun,

I'm glad you got it working, but now I'm confused.  The docker container
should already use the correct version of gcc (multilib).  Furthermore,
as I understand it, installing a different compiler on your machine
should not have any effect on the docker container.  Are you still using
the docker container?

Thanks,
Chris

On Thu, Jan 19, 2017 at 11:55:52PM +0800, Li-Chun Ko wrote:
> Hi Fabio,
> 
> After I install gcc-multilib all problems solved!
> I guess it is worthwhile to add a note for this in the tutorial.
> Thank you very much.
> 
> BTW, I am using Ubuntu 16.04.1 64 bit version.
> 
> Br,
> Lichun
> 
> 
> 2017-01-19 22:44 GMT+08:00 Li-Chun Ko :
> 
> > Hi all,
> >
> > I tried "$newt test @repos/apache-mynewt-core/kernel/os" in Linux but it
> > still doesn't work.
> > I also attached the error message of executing "$newt test all" below.
> > It seems some files are missing.
> >
> > Br,
> > Lichun
> >
> > 1. Error message when executing $newt test @repos/apache-mynewt-core/
> > kernel/os:
> >
> > lcko@lcko-VirtualBox:~/dev/go/myproj$ $newt test
> > @repos/apache-mynewt-core/kernel/os
> > panic: runtime error: invalid memory address or nil pointer dereference
> > [signal 0xb code=0x1 addr=0x0 pc=0x6879bf]
> >
> > goroutine 1 [running]:
> > panic(0x963920, 0xc82000e170)
> > /usr/lib/go-1.6/src/runtime/panic.go:481 +0x3e6
> > mynewt.apache.org/newt/newt/repo.(*Repo).Name(0x0, 0x1c, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/repo/repo.go:290
> > +0x1f
> > mynewt.apache.org/newt/newt/pkg.(*Dependency).setRepoAndName(0xc82043cbe0,
> > 0x7fcd8a960200, 0x0, 0x7ffdb79b70e3, 0x1c, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/pkg/dependency.go:63
> > +0x147
> > mynewt.apache.org/newt/newt/pkg.(*Dependency).Init(0xc82043cbe0,
> > 0x7fcd8a960200, 0x0, 0x7ffdb79b70e3, 0x1c, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/pkg/dependency.go:74
> > +0x55
> > mynewt.apache.org/newt/newt/pkg.NewDependency(0x7fcd8a960200, 0x0,
> > 0x7ffdb79b70e3, 0x1c, 0xd0dc40, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/pkg/dependency.go:84
> > +0x7f
> > mynewt.apache.org/newt/newt/project.(*Project).ResolvePackage(0xc820054540,
> > 0x7fcd8a960200, 0xc820070510, 0x7ffdb79b70dc, 0x23, 0x4, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/project/
> > project.go:522 +0x448
> > mynewt.apache.org/newt/newt/cli.testRunCmd(0xc82042cd80, 0xc820435210,
> > 0x1, 0x1)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/cli/
> > build_cmds.go:225 +0x140e
> > mynewt.apache.org/newt/newt/vendor/github.com/spf13/cobra.
> > (*Command).execute(0xc82042cd80, 0xc8204351d0, 0x1, 0x1, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/vendor/
> > github.com/spf13/cobra/command.go:636 +0x8e9
> > mynewt.apache.org/newt/newt/vendor/github.com/spf13/cobra.
> > (*Command).ExecuteC(0xc820075440, 0xc82042cd80, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/vendor/
> > github.com/spf13/cobra/command.go:722 +0x55c
> > mynewt.apache.org/newt/newt/vendor/github.com/spf13/cobra.
> > (*Command).Execute(0xc820075440, 0x0, 0x0)
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/vendor/
> > github.com/spf13/cobra/command.go:681 +0x2d
> > main.main()
> > /home/lcko/dev/go/src/mynewt.apache.org/newt/newt/newt.go:146 +0x2a2
> > exit status 2
> >
> >
> > 2. Error message when executing $newt test all:
> >
> > lcko@lcko-VirtualBox:~/dev/go/myproj$ cd repos
> > lcko@lcko-VirtualBox:~/dev/go/myproj/repos$ $newt test all
> > Testing package @apache-mynewt-core/boot/boot_serial/test
> > Compiling boot_serial.c
> > In file included from /usr/include/assert.h:35:0,
> >  from boot_serial.c:19:
> > /usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or
> > directory
> > compilation terminated.
> > Testing package @apache-mynewt-core/boot/bootutil/test
> > Compiling bootutil_misc.c
> > In file included from /usr/include/assert.h:35:0,
> >  from bootutil_misc.c:20:
> > /usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or
> > directory
> > compilation terminated.
> > Testing package @apache-mynewt-core/crypto/mbedtls/test
> > Compiling bootutil_misc.c
> > In file included from /usr/include/assert.h:35:0,
> >  from bootutil_misc.c:20:
> > /usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or
> > directory
> > compilation terminated.
> > Testing package @apache-mynewt-core/encoding/base64/test
> > Compiling bootutil_misc.c
> > In file included from /usr/include/assert.h:35:0,
> >  from bootutil_misc.c:20:
> > /usr/include/features.h:367:25: fatal error: sys/cdefs.h: No such file or
> > directory
> > compilation terminated.
> > Testing package @apache-mynewt-core/encoding/json/test
> > Compiling bootutil_misc.c
> > In file included from /usr/include/assert.h:35:0,
> >  

Re: MBUF sizing for the bluetooth stack

[Christopher Collins]

On Thu, Jan 19, 2017 at 10:57:58AM -0800, Christopher Collins wrote:
> On Thu, Jan 19, 2017 at 03:46:49AM -0800, Simon Ratner wrote:
> > A related question: how does this map to large ATT_MTU and fragmented
> > packets at the L2CAP level (assuming no data length extension)? Does each
> > fragment get its own mbuf, which are then chained together, or does the
> > entire packet get reassembled into a single mbuf if there is room?
> 
> If the host needs to send a large packet, it packs the payload into an
> mbuf chain.  By "packs," I mean each buffer holds as much data as
> possible with no regard to the maximum L2CAP fragment size.
> 
> When the host sends an L2CAP fragment, it splits the fragment payload
> off from the front of the mbuf chain, constructs an ACL data packet, and
> sends it to the controller.  If a buffer at the front of mbuf can be
> freed, now that data has been removed, the host frees it.
> 
> If you are interested, the function which handles fragmentation and
> freeing is mem_split_frag() (util/mem/src/mem.c).

I rushed this response a bit, and there are some important details I
neglected.

* For the final L2CAP fragment in a packet, the host doesn't
do an allocating or copying.  Instead, it just prepends an ACL data
header to the mbuf chain and sends it to the controller.

* For all L2CAP fragments *other than the last*, the host allocates an
additional mbuf chain to hold the ACL data packet.  The host then copies
the fragment data into this new chain, sends it, and frees buffers from
the front of the original chain if possible.  The number of buffers that
get allocated for the fragment depends on how the maximum L2CAP fragment
size compares to the msys mbuf size.  If an msys mbuf buffer has
sufficient capacity for a maximum size L2CAP fragment, then only one
buffer will get allocated.  If the mbuf capacity is less, the chain that
gets allocated will consist of multiple buffers.

* An L2CAP fragment mbuf chain contains the following:
* mbuf pkthdr   (8 bytes)
* HCI ACL data header   (4 bytes)
* Basic L2CAP header(4 bytes)
* Payload   (varies)

* For incoming data, the host does not do any packing.  Each L2CAP
fragment is simply chained together.

Re: MBUF sizing for the bluetooth stack

[Christopher Collins]

On Thu, Jan 19, 2017 at 03:46:49AM -0800, Simon Ratner wrote:
> A related question: how does this map to large ATT_MTU and fragmented
> packets at the L2CAP level (assuming no data length extension)? Does each
> fragment get its own mbuf, which are then chained together, or does the
> entire packet get reassembled into a single mbuf if there is room?

If the host needs to send a large packet, it packs the payload into an
mbuf chain.  By "packs," I mean each buffer holds as much data as
possible with no regard to the maximum L2CAP fragment size.

When the host sends an L2CAP fragment, it splits the fragment payload
off from the front of the mbuf chain, constructs an ACL data packet, and
sends it to the controller.  If a buffer at the front of mbuf can be
freed, now that data has been removed, the host frees it.

If you are interested, the function which handles fragmentation and
freeing is mem_split_frag() (util/mem/src/mem.c).

Chris

Using Eclipse for Mynewt development

[Christopher Collins]

Here are some of my notes for getting Mynewt to work with eclipse.  I
wrote these a while back, so there may be a few inaccuracies.  Also,
these notes assume you are building bletiny for the nRF52dk BSP, so you
may need to adjust accordingly.

Thanks,
Chris

*** Setup

1. Download Eclipse Neon (https://www.eclipse.org/downloads/).

2. Start Eclipse.

3. Eclipse -> Preferences -> C/C++ -> Environment; Add a "PATH" Variable
   with the value "/usr/local/bin" (or wherever arm-none-eabi-gdb is
   located on your machine).

4. In your newt project directory, overwrite
   repos/apache-mynewt-core/hw/bsp/nrf52dk/nrf52dk_debug.sh with the
   attached version.

*** New project

5. File -> New -> New Project -> C/C++ -> C Project ->
   Makefile project -> Empty Project

6. Select "-- Other Toolchain --" in the Toolchains panel.

7. Type your project's name in the "Project name" field.

8. Select your newt project directory in the "Location" field.

(see "new-project.png").

9. Click Finish.

*** Debug configuration

10. Right-click your project and select "Debug As..." ->
"Debug Configurations".

11. Double click "C/C++ Remote Application" to create a new debug
configuration.  The new configuration should now be selected.

12. At the bottom of the window, you should see: "Using GDB (DSF)
Automatic Remote Debugging Launcher - Select other...".  Click the
"Select other..." link.

13. Check "Use configuration specific settings"

14. Select "GDB (DSF) Manual Remote Debugging Launcher"

(see "select-launcher.png").

15. Click OK.

16. (Main tab) In the "C/C++ Application" field, select the .elf file
corresponding to your newt target.

17. (Main tab) Check "Disable auto build"

(see "debug-main.png").

18. (Debugger, Main tab) In the "GDB debugger" field, type:
arm-none-eabi-gdb

(see "debug-debugger-main.png").

19. (Debugger, Connection tab) In the "Port number" field, type: 

(see "debug-debugger-connection.png").

20. Click Apply and Close.

*** Debugging

21. In a shell, run "newt run  0".  JLinkGDBServer should
start up and listen for a connection from gdb.

21. Select your project in the Project Explorer.

22. Click the icon with a picture of a bug.  The debugger should start.

23. The board will be in a bad state and requires a reset.  Type
"monitor reset" in the Console pane

(see "monitor-reset.png")

24. Click the Resume button (play button with a yellow rectangle on its
left).  The program should now halt at the start of main().


nrf52dk_debug.sh
Description: Bourne shell script

Re: stopping scan & adv in bleprph example

[Christopher Collins]

Hi, sorry for the late response.  Somehow I missed this.

On Mon, Jan 16, 2017 at 08:22:27PM -0800, will sanfilippo wrote:
> Yes, Mynewt works the same way as FreeRTOS in this respect. Well, at least in 
> the way you are describing FreeRTOS. We have a tickless OS and when we decide 
> to go to sleep we are waiting for an interrupt to wake us up.
> 
> Regarding the radio: there are some registers that are only programmed once, 
> so if you switch to your own custom RF stack and you want to switch back to 
> bluetooth, you would either have to write some custom code or reset the link 
> layer. There is an API to do this but I am not sure if it is accessible to 
> the application developer.

It's not a part of the published host API, but the following function
should cause the host to reset the controller:

void ble_hs_sched_reset(int reason)

The reason code doesn't matter; it is just something the host uses when
reporting terminated connections.  You can just specify 1.

Since this function isn't part of the API, you'll need to declare a
prototype by hand:

void ble_hs_sched_reset(int reason);

Thanks,
Chris

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

On Tue, Jan 17, 2017 at 09:06:36PM -0700, Jacob Rosenthal wrote:
[...]
> As you can see, I cant seem to get slot1 active and confirmed

It looks like one of two things is going wrong:

1. The loader (bleprph) is failing to jump into the application
   (splitty), or

2. The application (splitty) is crashing immediately.

Do you have the latest newt tool from develop?  There was a bug in the
newt tool which would cause issue 2, but it was fixed a few days ago.
If I recall the bug, splitty's sysinit was initializing the the host
package, but not the HCI transport package, causing an invalid memory
access when host started up.

Anyway, if you don't have the latest newt, try pulling from develop and
reinstalling.  Otherwise, I think the next step is to debug this in gdb.
Feel free to reach out here or on IRC.

Thanks,
Chris

Re: stopping scan & adv in bleprph example

[Christopher Collins]

Hi Chew,

On Tue, Jan 17, 2017 at 01:08:16AM +, Lm Chew wrote:
> So calling ble_gap_adv_stop and ble_gap_disc_cancel will stop all radio 
> activity is that correct?

It depends on what your application has told the stack to do.  If you
have initiated a connection, you will also need to cancel that:

ble_gap_adv_stop()
ble_gap_disc_cancel()
ble_gap_conn_cancel()

If any connections have already been established, then they will also
cause radio activity.  You can terminate connections with
ble_gap_terminate().

Chris

> Is it safe to modify the Radio setting(on the physical just like in ble_phy) 
> after just calling these functions?
> 
> Hi Will,
> 
> Not exactly a "system off" I am looking for.
> Previously I am using FreeRTOS tickless mode where the MCU will remain in 
> sleep mode most of the tire unless there is a task to perform.
> 
> I am asking this because in the bleprph example I don't see any function 
> being called to put the MCU to sleep.
> 
> Does mynewt OS work the same way as FreeRTOS?
> 
> Best Regards,
> Chew
> 
> 
> 
> 
> 
> On Tue, Jan 17, 2017 at 1:57am, will sanfilippo 
> <wi...@runtime.io<mailto:wi...@runtime.io>> wrote:
> 
> If by deep sleep you mean "system off" mode requiring some form of wakeup, it 
> is curently not implemented. You would have to hook that in yourself.
> 
> > On Jan 16, 2017, at 9:22 AM, Christopher Collins <ccoll...@apache.org> 
> > wrote:
> >
> > Hi Chew,
> >
> > On Mon, Jan 16, 2017 at 11:33:23AM +, Lm Chew wrote:
> >> Hi,
> >>
> >> How do I stop the scan &  adv in the bleprph example.
> >>
> >> I tried calling the ble_ll_scan_sm_stop(1) and  ble_ll_adv_stop in my app, 
> >> but I am still able to see the device on my phone when I perform a scan.
> >
> > To stop advertising, call: ble_gap_adv_stop()
> > (http://mynewt.apache.org/latest/network/ble/ble_hs/ble_gap/functions/ble_gap_adv_stop/)
> >
> > For BLE operations, an application should only use the host interface.
> > Functions with the "ble_ll" prefix are defined by the controller, not
> > the host, so your application should not call them.
> >
> > Regarding scanning- the bleprph app doesn't perform any scanning, so
> > there is no need to stop scanning.  This application only implements the
> > peripheral role, so operations like scanning and initiating a connection
> > are not compiled in.  However, if you have a different app which does
> > support scanning, you would stop the scan procedure by calling
> > ble_gap_disc_cancel()
> > (http://mynewt.apache.org/latest/network/ble/ble_hs/ble_gap/functions/ble_gap_disc_cancel/)
> >
> >> I am trying to switch between my custom rf stack  and nimble bt stack. So 
> >> I need to disable nimble  operation before running my custom RF Stack.
> >> And once I am done what I need using the custom RF Stack, I will switch 
> >> back nimble.
> >>
> >> Another question, how do you put the MCU to deep sleep while using nimble 
> >> stack? In the example the MCU does not goes to deep sleep.
> >
> > Sorry, I am not sure about this one.  I am not sure this is actually
> > supported yet, but I'll let someone more knowledgable chime in.
> >
> > Chris
> 

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Just a follow up for the list-

It turns out there were a number of issues with split images.  I believe
they have all been resolved now.  Some of the required changes were
non-trivial, so I wanted to summarize what I did.

1. bleprph doesn't jump to app (slot 2).

This one is straightforward.  The bleprph app wasn't written to behave
as a loader.  Rather, it always started the OS, even when configured to
run the second stage app.  The fix was just to copy and paste the
generic loader code from slinky.

2. Jump from loader to app required too much RAM.

When a loader transitions to the second stage app, it executes a
modified version of the boot loader.  The boot loader was using quite a
bit of RAM for caching information about the layout of sectors within
the image slots.  In normal, non-split, operation, such RAM usage isn't
a problem because there is no OS running, and therefore no other code to
share memory with.  In the modified version that runs during the split
transition, other code *is* running, so RAM for the sector cache gets
allocated via malloc().  About 4kB was getting malloced here, which was
failing for me when running on the 16 kB RAM nRF51dk BSP.  The fix is to
not cache the sector data, but to read it when needed.

3. Loader-only packages were not getting initialized in the second stage
app.

This was a bug in the newt tool.  When generating the sysinit_app()
function, the tool excluded packages which are not explicitly referenced
in the app's dependency tree.  This is wrong; all packages should get
initialized in the second stage app.

Thanks,
Chris


On Fri, Jan 13, 2017 at 04:38:25PM -0700, Jacob Rosenthal wrote:
> Ah, I expected gdb of optimized split code to be a nightmare and hadnt
> tried yet.
> newtmgr has never crashed.. it just hangs forever
> 
> I cant get the SOFT anymore now. hrmmm Just hanging out here..
> (gdb) where
> #0  os_tick_idle (ticks=1189) at
> repos/apache-mynewt-core/hw/mcu/nordic/nrf51xxx/src/hal_os_tick.c:158
> #1  0x8c1e in os_idle_task (arg=) at
> repos/apache-mynewt-core/kernel/os/src/os.c:110
> #2  0x in ?? ()
> (gdb) c
> 
> Any chance I could impose on you to debug in irc with me sometime?
> 
> 
> On Thu, Jan 12, 2017 at 9:42 PM, Christopher Collins <ccoll...@apache.org>
> wrote:
> 
> > Hi Jacob,
> >
> > If that reboot log entry is recent, it looks like your device is
> > crashing.  A reason of "SOFT" indicates a firmware crash [*].  I would
> > say the quickest way to debug this is to run the code in gdb.  You can
> > do this with the "newt run" command:
> >
> > newt run  0
> >
> > When gdb comes up, type c and press enter.  Then in a different shell,
> > send a newtmgr command to the device.  If device crashes, gdb should
> > indicate such.  If Mynewt crashes, can you please type the following
> > commands in gdb:
> >
> > bt
> > p *g_current_task
> > p os_msys_init_1_mempool
> >
> > And send the output?
> >
> > Thanks,
> > Chris
> >
> > [*] Augmenting the reboot log entry with the line number and filename
> > where the crash occurred is on the to do list.
> >
> >
> > On Thu, Jan 12, 2017 at 08:41:03PM -0700, Jacob Rosenthal wrote:
> >
> >
> >
> > > turned off a ton more bluetooth shit for more ram
> > >
> > > both still hang:
> > > newtmgr -c serial1 logs log_list
> > > newtmgr -c serial1 image list
> > >
> > > but log in shell now has:
> > > newtmgr 4976:Dumping log reboot_log
> > >
> > >
> > > On Thu, Jan 12, 2017 at 8:33 PM, Jacob Rosenthal <
> > jakerosent...@gmail.com>
> > > wrote:
> > >
> > > > log on shell shows lots of these
> > > >
> > > > 9050:[0] rsn:SOFT, cnt:1, img:0.0.0.0
> > > >
> > > > On Thu, Jan 12, 2017 at 8:28 PM, Jacob Rosenthal <
> > jakerosent...@gmail.com>
> > > > wrote:
> > > >
> > > >> All newtmgr commands on serial are hanging indefinitely.
> > > >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr -c
> > serial1
> > > >> image list
> > > >> ^C
> > > >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr conn
> > show
> > > >> Connection profiles:
> > > >>   serial1: type=serial, connstring='/dev/tty.usbmodem1411'
> > > >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$
> > > >>
> > > >> Looks like it works for the splitty/slinky demo
> > > >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

On Fri, Jan 13, 2017 at 04:38:25PM -0700, Jacob Rosenthal wrote:
> Ah, I expected gdb of optimized split code to be a nightmare and hadnt
> tried yet.
> newtmgr has never crashed.. it just hangs forever
> 
> I cant get the SOFT anymore now. hrmmm Just hanging out here..
> (gdb) where
> #0  os_tick_idle (ticks=1189) at
> repos/apache-mynewt-core/hw/mcu/nordic/nrf51xxx/src/hal_os_tick.c:158
> #1  0x8c1e in os_idle_task (arg=) at
> repos/apache-mynewt-core/kernel/os/src/os.c:110
> #2  0x in ?? ()
> (gdb) c

It would be interesting to see how many mbufs are free:

p os_msys_init_1_mempool

> Any chance I could impose on you to debug in irc with me sometime?

Sure.  I'm on IRC, so just shout any time.

Chris

Re: Supporting multiple filesystems and "block" devices

[Christopher Collins]

On Fri, Jan 13, 2017 at 08:29:33PM -0200, Fabio Utzig wrote:
> Yes, I was expecting it! So I had 3 options in mind:
> 
> Option 1 - Use a "drive" name as was implemented. Btw, it's not limited
> to single letter drive names! Sure if someone really likes DOS, they
> could use A:, B:, etc. This choice is extremely easy to parse. And it
> also brings me fond memories of AmigaDOS!
> 
> Option 2 - Use a directory as a mount point like Unix but without
> implementing a full tree struct. Also very easy to parse, but somehow I
> find it very non-elegant because there would be no root path and such. I
> just personally dislike it because it looks incomplete, which brings me
> to:
> 
> Option 3 - Use a proper tree structure with mount points, mounts inside
> other mount's directories, etc. This is quite harder to do properly but
> would be my preferred aesthetic choice. But this needs extra things like
> ensuring the ordering of mounts, more error handling (if a root mount
> fails for any reason what to do with the remaining drives, etc). I would
> like to move to this but it could be added later because it's basically
> adding code and what is in place stays there, I think. 
> 
> If you think a tree structure is a must and "drive" names are an
> aberration, I would be OK at giving a try on implementing a proper VFS!

Re: option 2- I understand what you mean about the missing root path,
and I agree that it would be odd.  Personally, I don't think option 3 is
worth the effort, at least not now.  Just speaking for myself, I think
option 1 is not a real problem.  Like I said, it was just a knee-jerk
reaction when I saw the colon!

Thanks,
Chris

Re: [RFC] Refactor UUID handling in Nimble

[Christopher Collins]

Hi Andrzej,

That all sounds good to me.

Thanks,
Chris

On Fri, Jan 13, 2017 at 02:58:19PM +0100, Andrzej Kaczmarek wrote:
> Hi,
> 
> I've refactored Nimble code to use dedicated type for UUID handling instead
> of generic byte-array. Full diff is available here (it is split into
> several patches for now to separate changes in stack and apps):
> 
> https://github.com/andrzej-kaczmarek/incubator-mynewt-core/c
> ompare/develop...andrzej-kaczmarek:nimble/uuid
> 
> Start by looking at ble_uuid.h which defines new structures and helpers for
> UUIDs (or see below where I included essential part):
> 
> https://github.com/andrzej-kaczmarek/incubator-mynewt-core/b
> lob/1f571c7aeb22eb4e2ae96ca6d01d612afb338fd4/net/nimble/host
> /include/host/ble_uuid.h
> 
> Now, let me explain what it is all about:
> 
> Current implementation handles all UUIDs as long 128-bit values so they are
> passed like this (byte-array) in API and stored internally. This creates
> overhead for handling short 16-bit UUIDs in terms of both memory and
> processing time because:
> - short values are stored as long ones anyway,
> - they have to be converted back and forth since short UUID cannot be put
> as long one in ATT PDU,
> - there is no generic code to handle the above so similar patterns are
> repeated over and over again.
> 
> With new approach there is dedicated type to pass UUID which contains also
> its type so this information is available immediately - this is used e.g.
> BlueZ and Zephyr.
> 
> enum {
> BLE_UUID_TYPE_16 = 16,
> BLE_UUID_TYPE_32 = 32,
> BLE_UUID_TYPE_128 = 128,
> };
> 
> /* Generic UUID type, to be used only as a pointer */
> typedef struct {
> uint8_t type;
> } ble_uuid_t;
> 
> typedef struct {
> ble_uuid_t u;
> uint16_t value;
> } ble_uuid16_t;
> 
> typedef struct {
> ble_uuid_t u;
> uint32_t value;
> } ble_uuid32_t;
> 
> typedef struct {
> ble_uuid_t u;
> uint8_t value[16];
> } ble_uuid128_t;
> 
> /* Universal UUID type, to be used for any-UUID static allocation */
> typedef union {
> ble_uuid_t u;
> ble_uuid16_t u16;
> ble_uuid32_t u32;
> ble_uuid128_t u128;
> } ble_uuid_any_t;
> 
> This particular approach to handle UUIDs comes from Zephyr which uses neat
> trick to have common type for any UUID yet it allows to store 16-bit values
> using less memory than for 128-bit values. There are dedicated types for
> 16- and 128-bit UUIDs so memory for an UUID can be allocated, but the API
> uses "stub" type which defines only UUID type and is used as a pointer to
> an actual UUID variable. You can use helpers to get UUID value or compare
> them quickly without need to convert between 16- and 128-bit values. See
> following examples how this works (or see sample apps):
> 
> // 16-bit UUID value with initialization
> ble_uuid16_t uuid16 = BLE_UUID16_INIT(0x2801);
> // 128-bit UUID value with initialization
> ble_uuid128_t uuid128 = BLE_UUID128_INIT(0x00, 0x11, ... 0xFF);
> 
> // "stub" type is used to pass any type of UUID via API
> ble_uuid_t *uuid1 = 
> ble_uuid_t *uuid2 = 
> 
> // comparing UUIDs is really easy now
> if (ble_uuid_cmp(uuid1, uuid2) == 0) { /* equal */ )
> 
> // since in most cases we use 16-bit UUIDs, it's convenient to easily
> retrieve 16-bit value (or 0 if UUID is not 16-bit)
> uint16_t u16 = ble_uuid_u16(uuid1);
> 
> // the above is equivalent of
> u16 = uuid1->type == BLE_UUID_TYPE_16 ? BLE_UUID16(uuid1)->value : 0;
> 
> // we can easily define UUID inline as well, e.g. when defining service
> static const struct ble_gatt_svc_def ble_svc_gap_defs[] = {
> {
> .type = BLE_GATT_SVC_TYPE_PRIMARY,
> .uuid = BLE_UUID16_DECLARE(BLE_SVC_GAP_UUID16),
> (...)
> 
> // finally, if we need to store any UUID there is an union defined to help
> ble_uuid_any_t uuid;
> uuid.u16 = uuid16;
> 
> *IMPORTANT NOTE*:
> There are two major differences between old and new API:
> - Shen defining and registering services, in old API UUID value is copied
> to attribute structure so it does not matter where UUID passed as parameter
> is defined. In new API only pointer to UUID is stored thus you cannot e.g.
> define value on stack since it will be destroyed.
> - You need to make sure that 16-bit UUIDs are defined using helpers
> dedicated for 16-bit UUIDs only, i.e. you should not create 128-bit UUID
> with Bluetooth base since it will not be detected as 16-bit UUID.
> 
> Finally, there are of course some savings in both flash and RAM:
> 
> before:
> < 136 286 *fill*
> <   307385027 apps_bletiny.a
> <   536333411 net_nimble_host.a
> < 880 290 net_nimble_host_services_ans.a
> < 496 289 net_nimble_host_services_gap.a
> < 306  86 net_nimble_host_services_gatt.a
> <  160884   2784  15828 179496  2bd28
> /home/andk/devel/mynewt/bletiny/bin/targets/bletiny/
> app/apps/bletiny/bletiny.elf
> 
> after:
> > 143 364 *fill*
> >   304705053 apps_bletiny.a
> >   534093415 net_nimble_host.a
> >   

Re: Supporting multiple filesystems and "block" devices

[Christopher Collins]

Hi Fabio,

Sounds great.  I only have one comment:

On Fri, Jan 13, 2017 at 03:09:48PM -0200, Fabio Utzig wrote:
> > All the fs_* functions where updated to support multiple disks. To
> > access a file/dir now will required adding the name of the disk as a
> > prefix like this:
> > 
> > rc = fs_opendir("mmc0:/", );
> > rc = fs_open("flash0:/testfile.txt", FS_ACCESS_READ, );

I'm sure you anticipated this comment :). Seeing a ':' in a path brings
back horrible memories for me.  I am not sure if this is just an
aesthetic judgment, or if the "UNIX way" would actually be better.  If
it's just an asthetic thing, then I'll get over it, but I can't help but
think there is code out there that would manage better with UNIX paths
rather than DOS paths.

I haven't looked at the code yet, so I don't know how much easier the
use of the colon character makes things.  What do you think about using
"/flash0/testfile.txt" rather than "flash0/textfile.txt"?

Chris

Re: Override pin settings

[Christopher Collins]

On Fri, Jan 13, 2017 at 11:35:13AM -0500, David G. Simmons wrote:
> Here's the entire syscfg.yml file:
> 
> ### Package: targets/air_q
> 
> syscfg.vals:
> OPENOCD_DEBUG: 1
> # Enable the shell task.
> SHELL_TASK: 1
> STATS_CLI: 1
> 
> CONSOLE_TICKS: 1
> CONSOLE_PROMPT: 1
> 
> UART_0_PIN_TX:
> description: 'New Pin Assignment'
> value: 23
> UART_0_PIN_RX:
> description: 'New Pin Assignment'
> value: 24
> 
> So I was trying to override them in the syscfg.vals section. And that is what 
> fails. the old value from the package's syscfg.yml is removed, but the new 
> value isn't substituted in its place.

The last two entries should look like this:

UART_0_PIN_TX: 23
UART_0_PIN_RX: 24

I think this will be a common mistake, so we should look at making newt
report an error here.

Thanks,
Chris

Re: Override pin settings

[Christopher Collins]

Hi David,

On Fri, Jan 13, 2017 at 09:52:35AM -0500, David G. Simmons wrote:
> Before I file a bug in Jira on this, I thought I'd ask if this is expected 
> behavior:
> 
> From my project's syscfg.yml:
> UART_0_PIN_TX:
> description: 'New Pin Assignment'
> value: 23
> UART_0_PIN_RX:
> description: 'New Pin Assignment'
> value: 24
> 
> Results in newt target config show air_q:
> 
> * Setting: UART_0_PIN_RX
> * Description: TBD
> * Value:
> * Overridden: targets/air_q, default=5
>   * Setting: UART_0_PIN_TX
> * Description: TBD
> * Value:
> * Overridden: targets/air_q, default=6
> 
> So it picks up that the value was overridden, but it doesn't pick up the 
> actual values. It looks like I have to go into 
> hw/bsp/arduino_primo_nrf52/syscfg.yml and change it there for any effect to 
> be seen.

It does look like there is a bug here.  Newt should report an error when
it encounters the syscfg file.  You may already know all this, but for
others' benefit- a syscfg.yml file can do two things:
1. Define new settings
2. Override existing settings

You define new settings with the "syscfg.defs" mapping, e.g.,

syscfg.defs:
UART_1:
description: 'Bitbanger UART'
value:  0

You override existing settings with "syscfg.vals", e.g.,

syscfg.vals:
UART_1: 1

That is, the description, value, and other fields are specified at the
point of definition.  When a setting is overridden, only the new value
gets specified.

>From your email, it looks like the target's syscfg.yml is redefining
existing settings, which should cause newt to raise an error.  I'll look
into this error reporting issue, but in the meantime, I would change
those UART pin overrides to use the syscfg.vals mapping rather than
syscfg.defs.

Chris

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

If that reboot log entry is recent, it looks like your device is
crashing.  A reason of "SOFT" indicates a firmware crash [*].  I would
say the quickest way to debug this is to run the code in gdb.  You can
do this with the "newt run" command:

newt run  0

When gdb comes up, type c and press enter.  Then in a different shell,
send a newtmgr command to the device.  If device crashes, gdb should
indicate such.  If Mynewt crashes, can you please type the following
commands in gdb:

bt
p *g_current_task
p os_msys_init_1_mempool

And send the output?

Thanks,
Chris

[*] Augmenting the reboot log entry with the line number and filename
where the crash occurred is on the to do list.


On Thu, Jan 12, 2017 at 08:41:03PM -0700, Jacob Rosenthal wrote:



> turned off a ton more bluetooth shit for more ram
> 
> both still hang:
> newtmgr -c serial1 logs log_list
> newtmgr -c serial1 image list
> 
> but log in shell now has:
> newtmgr 4976:Dumping log reboot_log
> 
> 
> On Thu, Jan 12, 2017 at 8:33 PM, Jacob Rosenthal <jakerosent...@gmail.com>
> wrote:
> 
> > log on shell shows lots of these
> >
> > 9050:[0] rsn:SOFT, cnt:1, img:0.0.0.0
> >
> > On Thu, Jan 12, 2017 at 8:28 PM, Jacob Rosenthal <jakerosent...@gmail.com>
> > wrote:
> >
> >> All newtmgr commands on serial are hanging indefinitely.
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr -c serial1
> >> image list
> >> ^C
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr conn show
> >> Connection profiles:
> >>   serial1: type=serial, connstring='/dev/tty.usbmodem1411'
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$
> >>
> >> Looks like it works for the splitty/slinky demo
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$ newtmgr -c serial1
> >> image list
> >> Images:
> >>  slot=0
> >> version: 0.0.0
> >> bootable: true
> >> flags: active confirmed
> >> hash: 21439de02cbf31626856374f44cbd490fd6def3ce3062b63d55ed2c19a8b
> >> 2b83
> >>  slot=1
> >> version: 0.0.0
> >> bootable: false
> >> flags:
> >> hash: 8b64ea89bf0495c0ccb25b96b3a7f06fd5e540e221f9659f9bc6b0d0d303
> >> d6f1
> >> Split status: matching
> >> Jacobs-MacBook-Air:mynewt-hr-observer jacobrosenthal$
> >>
> >> Ram issue? Whats a good way to see a log of the failed attempt since I
> >> have the shell?
> >>
> >>
> >>
> >> On Thu, Jan 12, 2017 at 7:51 PM, Christopher Collins <ccoll...@apache.org
> >> > wrote:
> >>
> >>> On Thu, Jan 12, 2017 at 07:06:37PM -0700, Jacob Rosenthal wrote:
> >>> > Still ok...  and Im able to interact with shell over serial and I
> >>> think its
> >>> > advertising!? Not sure why my newtmgr wont connect then.. Ideas to
> >>> >  troubleshoot?
> >>>
> >>> So you are sending newtmgr commands over serial?  Do all commands fail,
> >>> or just image list?
> >>>
> >>> > Also I was digging and found someone already PRed a ble transport for
> >>> > newtmgr https://github.com/apache/incubator-mynewt-core/pull/73/files
> >>> > but the api is a bit different now.. Anyone using that?
> >>>
> >>> Yes, the newtmgr BLE characteristic is being used and should work.  The
> >>> newtmgr command line tool has rudimentary support for BLE, but only when
> >>> run on linux.
> >>>
> >>> Thanks,
> >>> Chris
> >>>
> >>
> >>
> >

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

On Thu, Jan 12, 2017 at 07:06:37PM -0700, Jacob Rosenthal wrote:
> Still ok...  and Im able to interact with shell over serial and I think its
> advertising!? Not sure why my newtmgr wont connect then.. Ideas to
>  troubleshoot?

So you are sending newtmgr commands over serial?  Do all commands fail,
or just image list?

> Also I was digging and found someone already PRed a ble transport for
> newtmgr https://github.com/apache/incubator-mynewt-core/pull/73/files
> but the api is a bit different now.. Anyone using that?

Yes, the newtmgr BLE characteristic is being used and should work.  The
newtmgr command line tool has rudimentary support for BLE, but only when
run on linux.

Thanks,
Chris

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

On Thu, Jan 12, 2017 at 07:19:11PM -0700, Jacob Rosenthal wrote:
> OH really interesting.. In the updated documentation the biggest change
> from what I was doing was splitty is the app and bleprph is the loader. Is
> that intentional?

Yes - the loader needs to contain everything required for an image
upgrade.  The bleprph app contains the BLE stack and the newtmgr /
imgmgr server.  These components are what is required for an
over-the-air upgrade, so it makes sense to use bleprph as the loader.

Chris

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

On Mon, Jan 09, 2017 at 10:48:48AM -0800, Christopher Collins wrote:
> The boot sequence and upgrade procedure for the split image setup is a
> bit complicated.  I have been working on some documentation for this
> area, but haven't quite finished it.  I hope to have something ready in
> a day or two.  In the meantime, please feel free to ask any follow up
> questions.

FYI - The split image documentation has been updated on the Mynewt site:
http://mynewt.apache.org/latest/os/modules/split/split/

Chris

Re: How to change the CPU time frequency of mynewt and nimble stack

[Christopher Collins]

Hi Jiacheng,

On Wed, Jan 11, 2017 at 10:47:24AM +0800, WangJiacheng wrote:
> More information about nimble stack with 2 MHz CPU frequency. nimble-bleprph 
> can be connected by LightBlue, but after several seconds, it is disconnected 
> with message “Disconnected Alert: The peripheral has disconnected.”  With 4 
> MHz CPU frequency, nimble-bleprph can not be scanned by LightBlue.

Could you tell me more about the device running LightBlue?  At one
point, CoreBluetooth (the Apple Bluetooth library) had a bug that was
triggered when the NimBLE controller initiated the data length extension
procedure.  When this happened, the Apple device would send a bad
response, and the NimBLE device would disconnect.  If you are using an
older version of MacOS or iOS, then this could be the problem.

You can try disabling the data length extension feature to see if that
is the problem.  Do this by adding the following text to
targets//syscfg.yml:

syscfg.vals:
  BLE_LL_CFG_FEAT_DATA_LEN_EXT: 0

Thanks,
Chris

Re: How to change the CPU time frequency of mynewt and nimble stack

[Christopher Collins]

Hi Jiacheng,

I think your version of newt is still slightly out of date.  You can
install the latest as follows:

cd $GOPATH/src/mynewt.apache.org/newt/newt &&
   git checkout develop&&
   git pull origin develop &&
   go install  ;
   cd -


Thanks,
Chris

On Wed, Jan 11, 2017 at 09:04:05AM +0800, WangJiacheng wrote:
> Sterling,
> 
> Thanks.
> 
> Yes, the newt is already updated. “newt version” has return "Apache Newt 
> (incubating) version: 1.0.0-dev”.
> 
> Best Regards,
> 
> Jiacheng
> 
> > 在 2017年1月11日，08:58，Sterling Hughes  写道：
> > 
> > Hi Jiacheng,
> > 
> > You need to update your newt tool along with the new develop.
> > 
> > Best,
> > 
> > Sterling
> > 
> > On 10 Jan 2017, at 16:46, WangJiacheng wrote:
> > 
> >> Hi, Will,
> >> 
> >> I need more help, I have an error message when compile the target.
> >> 
> >> I’m currently working on the release branch, so upgrade to dev branch by:
> >>1. change file project.yml from "vers: 0-latest” to "vers: 0-dev”
> >>2. upgrade to dev branch “newt upgrade”
> >> 
> >> Then compile the target by “newt build nrf52_boot”, an error message as:
> >> 
> >> Building target targets/nrf52_boot
> >> Compiling boot.c
> >> Archiving boot.a
> >> Compiling bootutil_misc.c
> >> Compiling image_ec.c
> >> Compiling image_ec256.c
> >> Compiling image_rsa.c
> >> Compiling image_validate.c
> >> Compiling loader.c
> >> Archiving bootutil.a
> >> Error: In file included from aes.c:29:0:
> >> /Users/jiachengwang/dev/myproj/repos/apache-mynewt-core/crypto/mbedtls/include/mbedtls/config.h:2522:10:
> >>  error: #include expects "FILENAME" or 
> >> #include MBEDTLS_USER_CONFIG_FILE
> >>  ^
> >> 
> >> it seems the config file "mbedtls/config_mynewt.h” define in 
> >> “crypto/mbedtls/pkg.yml” is missed.
> >> 
> >> Thanks,
> >> 
> >> Jiacheng
> >> 
> >> 
> >> 
> >> 
> >> 
> >>> 在 2017年1月10日，11:06，WangJiacheng  写道：
> >>> 
> >>> Thanks, Will.
> >>> 
> >>> There is an Internet connection issue to GitHub.com currently, I’ll 
> >>> update the code later.
> >>> 
> >>> Best Regards,
> >>> 
> >>> Jiacheng
> >>> 
> >>> 
>  在 2017年1月10日，10:10，will sanfilippo  写道：
>  
>  Hello:
>  
>  This issue should now be fixed in the latest development branch. Note 
>  that this is not working on the nrf51 platforms but since you were using 
>  nrf52 it should work.
>  
>  Let me know if you see any issues with it.
>  
>  
> > On Jan 8, 2017, at 6:20 PM, WangJiacheng  
> > wrote:
> > 
> > Hi, Will,
> > 
> > Thanks a lot for your reply.
> > 
> > Yes，the hardwear processor clock frequency of nRF52 (Cortex M4F) is 64 
> > MHz and can not be changed.
> > 
> > The reason of changing CLOCK_FREQ is that I want re-use the internal 
> > timing of mynewt already there with more accurate timing, by calling 
> > function "os_cputime_get32()”.  I’m trying to implement a (soft) IC 
> > card reader by nRF52 with mynewt OS and nimble stack running.
> > 
> > I am also considering to use an independent timer (NRF_TIMER3 or  
> > NRF_TIMER4) at the cost of about 0.1mA current. I already use 
> > NRF_TIMER2 to provide a 4 MHz clock signal output from GPIO of nRF52. 
> > By reading the source code of apache-mynewt-core, my understanding is 
> > that NRF_TIMER0 and NRF_TIMER1 is already used by mynewt OS and nimble 
> > stack, is my understanding correct?
> > 
> > Thanks,
> > 
> > Jiacheng
> > 
> >> 在 2017年1月9日，01:10，will sanfilippo  写道：
> >> 
> >> Those should be the only two parameters you need to configure. Must be 
> >> a bug in the controller :-)
> >> 
> >> I think it is worthwhile to point out that CLOCK_FREQ only changes the 
> >> units of os cputime; it does not affect the speed at which the 
> >> processor runs. At least, I could not see any other uses of 
> >> CLOCK_FREQ. So, these settings only affect the nimble stack and the 
> >> controller specifically (internal controller timing).
> >> 
> >> I am curious why you wanted to change this variable; what were you 
> >> trying to achieve?
> >> 
> >> Thanks for pointing this out; I will take a look to see why it is not 
> >> working.
> >> 
> >>> On Jan 7, 2017, at 10:48 PM, WangJiacheng  
> >>> wrote:
> >>> 
> >>> Hi,
> >>> 
> >>> The default CPU time frequency of Mynewt OS and Nimble stack is 1 
> >>> MHz, I try to change the CPU time frequency to be 2 MHz, I modified 
> >>> the related 2 config files:
> >>> configure file 

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

On Sun, Jan 08, 2017 at 01:04:54PM -0700, Jacob Rosenthal wrote:
> On Mon, Jan 2, 2017 at 2:46 PM, Christopher Collins <ccoll...@apache.org>
> wrote:
> 
> > Do you know how big your loader is currently?
> 
> 
> Well, could be anything based on how much Im trying to strip from the
> nimble stack at any given time. but Im generally fighting to fit it within
> one slot which on the nrf51 is 110kB.

Yeah, it is currently pretty tough to get the OS + BLE stack to fit in
110kB.  Hopefully we'll get a chance to reduce the size of NimBLE soon.

> I'm also not sure if it would buy
> > you much extra flash space.
> 
> At this point Its not even about buying space, as I now have my other 110Kb
> back from the second image slot.. What I dont have anymore is update
> capability, so its about restoring that capability.
> 
> 
> > Im somewhat unclear if the 'loader' app can/should also serve as the
> > 'bootloader', or if it was intended that a separate bootloader would also
> > be present in addition to loader and application?
> >
> > This question of mine got buried above.. in this scenario, the distinction
> between loader and bootloader gets weird... My thought is theyre merged?

Sorry, I must have missed that question.  The boot loader is present in
the split image setup.  Here is how the boot sequence would look for the
various image setups:

1. Single image
* no boot loader
* image at address 0
* hardware boots directly into image

2. Stub image
* boot loader at address 0
* hardware boots into boot loader
* boot loader jumps to image in slot 0

3. Unified image (dual bank)
* boot loader at address 0
* hardware boots into boot loader
* if required, boot loader swaps images in the two image slots
  (i.e., if an upgrade or a fallback is being performed).
* boot loader jumps to image in slot 0

4. Split image
* boot loader at address 0
* hardware boots into boot loader
* if required, boot loader swaps images in the two image slots
  (i.e., if an upgrade or a fallback *between two loaders* is being
  performed).
* boot loader jumps to image in slot 0
* loader in slot 0 jumps to "app" in slot 1.

The boot sequence and upgrade procedure for the split image setup is a
bit complicated.  I have been working on some documentation for this
area, but haven't quite finished it.  I hope to have something ready in
a day or two.  In the meantime, please feel free to ask any follow up
questions.

Thanks,
Chris

Re: Compile and build time, any way to speed up?

[Christopher Collins]

Hi Cris,

On Fri, Jan 06, 2017 at 12:54:03PM -0500, Cris Frusina wrote:
> Fixed the TCP issue by restarting Virtualbox, did an newt upgrade and a 
> newt install with the 0-dev in the project file.
> 
> Still missing something as -j is not recognized.

The -j option is only supported in the develop branch.  To get the
develop version of newt:

cd $GOPATH/src/mynewt.apache.org/newt/newt &&
   git checkout develop
   git pull origin develop
   go install

But be careful: the develop version of newt is not compatible with
Mynewt version 1.0.0-b1.  If you use newt develop, then you'll also need
to use Mynewt develop.  You can upgrade your Mynewt repositories to the
develop branch as follows:

1. Open your project.yml in an editor.
2. Change all the "vers" fields to 0-dev (they are probably
   currently set to 0-latest).
3. Run "newt upgrade"

If you ever want to revert back to 1.0.0-b1, you can reverse the above
process, i.e., set the vers fields to 0-latest and run "newt upgrade".

Chris

> 
> ~/newt version
> Apache Newt (incubating) version: 1.0.0-b1
> 
> ~/newt build -j 5 ble_usb
> Error: unknown shorthand flag: 'j' in -j
> Usage:
>newt build  [target-names...] [flags]
> 
> Global Flags:
>-l, --loglevel string   Log level (default "WARN")
>-o, --outfile stringFilename to tee output to
>-q, --quiet Be quiet; only display error output
>-s, --silentBe silent; don't output anything
>-v, --verbose   Enable verbose output when executing commands
> 
> Cris
> 
> 
> 
> On 1/6/2017 12:27 PM, David G. Simmons wrote:
> > That looks like your network isn't working from your Docker instance. If 
> > that's actually the case, it could also (possibly) explain the torturously 
> > long compile times. Maybe.
> >
> > dg
> >> On Jan 6, 2017, at 12:24 PM, Cris Frusina  wrote:
> >>
> >> David,
> >>
> >> I've tried what you suggested I'm getting this error:
> >>
> >> *apache-mynewt-core**
> >> **Error: Get 
> >> https://api.github.com/repos/apache/incubator-mynewt-core/contents/repository.yml?ref=master:
> >>  dial tcp: lookup api.github.com on 10.0.2.3:53: read udp 
> >> 172.17.0.2:38728->10.0.2.3:53: i/o timeout*
> >>
> >>
> >> My project file looks like this now:
> >>
> >> project.name: "my_project"
> >>
> >> project.repositories:
> >> - apache-mynewt-core
> >>
> >> # Use github's distribution mechanism for core ASF libraries.
> >> # This provides mirroring automatically for us.
> >> #
> >> repository.apache-mynewt-core:
> >> type: github
> >> vers: 0-dev
> >> user: apache
> >> repo: incubator-mynewt-core
> >>
> >>
> >> Am I missing something?
> >>
> >> Thanks,
> >> Cris
> >>
> >> On 1/6/2017 12:05 PM, David G. Simmons wrote:
> >>> You could do that, but if you don't want to remake the whole Docker 
> >>> instance, you should be able to simply edit the project.yml file and 
> >>> change the vers strin for repository.apache-mynewt-core to 0-dev and then 
> >>> run newt install
> >>>
> >>> Best regards,
> >>> dg
> >>>
>  On Jan 6, 2017, at 5:23 AM, Fabio Utzig  wrote:
> 
>  I'm not using the docker version myself, but I think you could try
>  rebuilding the docker image:
> 
>  https://github.com/runtimeinc/newt-docker 
>  
> 
>  Inside the Makefile, change the branch that is checked out from "-b
>  mynewt_1_0_0_b1_tag" to "-b develop":
> 
>  https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15 
>  
> 
>  Att,
>  Fabio Utzig
> >>> --
> >>> David G. Simmons
> >>> (919) 534-5099
> >>> Web  • Blog  • 
> >>> Linkedin  • Twitter 
> >>>  • GitHub 
> >>> /** Message digitally signed for security and authenticity.
> >>> * If you cannot read the PGP.sig attachment, please go to
> >>>   * http://www.gnupg.com/  Secure your email!!!
> >>>   * Public key available at keyserver.pgp.com 
> >>> **/
> >>> ♺ This email uses 100% recycled electrons. Don't blow it by printing!
> >>>
> >>> There are only 2 hard things in computer science: Cache invalidation, 
> >>> naming things, and off-by-one errors.
> >>>
> >>>
> >>>
> > --
> > David G. Simmons
> > (919) 534-5099
> > Web  • Blog  • 
> > Linkedin  • Twitter 
> >  • GitHub 
> > /** Message digitally signed for security and authenticity.
> > * If you cannot read the PGP.sig attachment, please go to
> >   * http://www.gnupg.com/  Secure your email!!!
> >   * Public key available at 

Re: Compile and build time, any way to speed up?

[Christopher Collins]

On Thu, Jan 05, 2017 at 09:00:11AM -0500, Cris Frusina wrote:
> Hi David,
> 
> Sounds like a pain!
> 
> So I got an Ubuntu VM running on the Windows machine and installed the myNewt 
> natively. I didn't really see much of a speed bump (needs a bit further 
> testing). I'll try a different computer later today. Worst case I'll find a 
> Mac to try it on. 

Just an note - I ended up adding multithreaded build support to newt
yesterday.  In the develop branch, you can enable it with the
"-j " command line option, e.g.,

newt -j 5 build my_blinky_sim

A word of warning, though: don't upgrade your newt to develop if you
are using Mynewt 1.0-b1 and aren't prepared to upgrade that as well.
Since the 1.0-b1 release, some backwards-compatible changes were made to
the newt tool.

Thanks,
Chris

Re: Compile and build time, any way to speed up?

[Christopher Collins]

On Thu, Jan 05, 2017 at 08:34:04AM -0500, David G. Simmons wrote:
> Well, I spent the entire day yesterday working on this, and unfortunately, I 
> don't have any real results. 
> 
> My only Windows setup is a VM, and it turns out that running a Docker 
> instance on a Windows VM on top of Mac OS X is a fairly unrewarding 
> experience in and of itself, and getting the mynewt stuff running in that 
> setup is, to a first approximation, not easily achieved. 

Ouch, yeah... I had a similar experience a while back, with a similar
amount of wasted time.  Eventually a I concluded that it just isn't
possible to run VirtualBox in a Windows VM (the docker solution uses
VirtualBox).

Chris

Re: Single Bank firmware update (ie nordic vendor style dfu)

[Christopher Collins]

Hi Jacob,

On Sun, Jan 01, 2017 at 11:10:01PM -0700, Jacob Rosenthal wrote:
> On low flash devices like nrf51822 FLASH_AREA_IMAGE_1 equal to
> FLASH_AREA_IMAGE_0 is improbable, which means dual bank is likewise
> improbable and as a result we've talked previously about how to shrink
> FLASH_AREA_IMAGE_1 down to a stub.
> 
> However, my current understanding is newtmgr/imagemgr implementation is
> dual bank, wherein the application saves received bytes into
> COREDUMP_FLASH_AREA (ie FLASH_AREA_IMAGE_1) and reboots allowing bootloader
> (apps/boot) to safely make the swap.

Your understanding sounds correct to me.  If you don't have enough flash
for a second image slot, then image upgrade isn't possible.

Just a clarification- for a split image, both slots don't need to be
exactly the same size; the slot sizes just need to meet these
requirements:

* Both slots are large enough to accommodate the loader.
* Second slot is large enough to accommodate the app.

In other words, if your app is larger than your loader, you only need to
budget for the extra space in the second slot.  Unfortunately, I don't
think this allowance helps in your case.  I think the problem you're
facing is that the loader (kernel + BLE stack) is too large.

> I don't believe theres a good way to do a single bank receive from within
> the running application, which would mean this has to be moved into the
> bootloader, which means it need to share access to nimble, which means a
> bootloader fork that includes splitty, nimble, newtmgr, imagemgr.
> 
> Im somewhat unclear if the 'loader' app can/should also serve as the
> 'bootloader', or if it was intended that a separate bootloader would also
> be present in addition to loader and application?
> 
> Thoughts on this architecture, am I missing anything?

Hmm... I do see what you mean, but that sounds like a pretty fundamental
change to how image management works.  I'm also not sure if it would buy
you much extra flash space.  The boot loader is already fairly stripped
down, and it doesn't contain much code that could be shared by the image
upgrade mechanism.

I don't want to be dismissive of your idea, but my thinking is that the
best way forward is to try to reduce the size of the BLE stack.  This is
something that has been on the radar for quite a while.  We haven't
really looked at this at all, so there is probably some easy savings to
be had.  I have been meaning to do a quick survey of the BLE host to
determine what is contributing to its code size.  That said, if you want
to look at incorporating upgrade functionality into the boot loader,
don't let me stop you :).

Do you know how big your loader is currently?

Thanks,
Chris

Re: MyNewt Mqueue not working

[Christopher Collins]

Hi Then,

On Mon, Jan 02, 2017 at 01:23:33PM +0800, then yon wrote:
> Dear Support,
> 
> I been using MyNewt v1.0.0-dev, and i tried to make use of Mqueue.
> 
> What i did was following the tutorial below:
> 
> https://mynewt.apache.org/os/core_os/mqueue/mqueue/
> Since the tutorial is not updated, i tried to modify it but unfortunately i 
> can't make it work.

[...]

Your code looks good.  There is just one crucial step missing: a task
needs to poll the eventq you created (queue_task_evq).  Just to clarify
how mqueues work: when os_mqueue_put() is called, two things happen:
1. The specified mbuf is placed on the mqueue.
2. The event associated with the mqueue is put on the specified
   eventq, but only if the event is not already enqueued.

If a task is listening on the eventq, then it can your mqueue processing
function when it sees the mqueue event.

I think you have two options:
1. Remove queue_task_evq from your code; use the other eventq
   (blinky_evq) with your mqueue.  blinky_evq is already being
   processed correctly by the shell_task_handler() function, so if
   you enqueue an mqueue event to it, your mqueue data will get
   processed.

2. Create an additional task which processes the queue_task_evq
   eventq.  This task's handler would probably look identical to
   shell_task_handler(), i.e., a tight loop that repeatedly calls
   os_eventq_run().

I recommend trying the simpler option 1 first.  Generally, it is best to
minimize the number of tasks your app uses so that it requires less RAM.

Thanks,
Chris

Re: Removing or shrinking flash areas

[Christopher Collins]

Darn, I forgot to mention a critical point.  To use the slot-1 stub
solution, you will probably need to build and upload a new boot loader.
Stub slot support was added to the develop branch on Dec. 21, 2016
(https://issues.apache.org/jira/browse/MYNEWT-520).

Chris

On Fri, Dec 30, 2016 at 03:39:40PM -0800, Christopher Collins wrote:
> Hi Jacob,
> 
> On Fri, Dec 30, 2016 at 03:28:08PM -0700, Jacob Rosenthal wrote:
> > I think deps are a part of the story, but yeah the goal is to successfully
> > get a target bsp back to 1 larger flash image, or as many as the common set
> > of deps require but shrunk way down.
> 
> So you just want to eliminate the need for a second image slot in the
> flash map?
> 
> There are four flash areas that are relevant here.  Just to be clear: a
> flash area is simply a named region of flash.  Each BSP's bsp.yml file
> defines its flash map as a sequence of flash areas.
> 
> Anyway, the four relevant flash areas are:
> * FLASH_AREA_BOOTLOADER:
> * FLASH_AREA_IMAGE_0:
> * FLASH_AREA_IMAGE_1:
> * FLASH_AREA_IMAGE_SCRATCH:
> 
> The boot loader area is placed at the board's start address in flash.
> That is, the device boots directly into the boot loader.  The boot
> loader then possibly swaps the two image slots and jumps to the image in
> slot 0.  This process is described in some detail in
> boot/bootutil/design.txt.
> 
> If your device only needs a single image, you have two options:
> 1. slot-1 stub
> 2. no boot loader
> 
> Option 1 is the simplest, but option 2 gives you more flash space.  I
> will describe both options below.
> 
> *** Slot-1 stub:
> 
> This option involves greatly reducing the size of the FLASH_AREA_IMAGE_1
> area.  When I did this with the nRF51dk, I used an area size of 2kB.  I
> think you shld be able to reduce it to the size of an individual
> sector (1kB), but I have not tried this.
> 
> There are two changes you need to make:
> * (bsp.yml)
> o Increase the size of FLASH_AREA_IMAGE_0.
> o Reduce the size of FLASH_AREA_IMAGE_1.
> o Increase the offset of FLASH_AREA_IMAGE_0.
> 
> * (nrf51xxac.ld)
> o Increase the size of the FLASH region.
> 
> I have attached modified versions of these two files that you can try.
> 
> *** No boot loader:
> 
> This option involves removing the boot loader entirely.
> Instead of a boot loader, the image binary is placed at the board's
> start address in flash such that the device boots directly into it.
> Without a boot loader, you can eliminate the following two areas from
> the flash map:
> * FLASH_AREA_BOOTLOADER:
> * FLASH_AREA_IMAGE_1:
> 
> Unfortunately, the scratch area cannot be eliminated at the moment,
> since it is required by the reboot log.  I think this is something we
> should address soon, but in the meantime, you are stuck with the scratch
> area.
> 
> It has been a while since I've tried running an image without a boot
> loader, so my memory might be a bit foggy.  I think the following steps
> are all you need:
> * (bsp.yml)
> o Remove FLASH_AREA_BOOTLOADER
> o Remove FLASH_AREA_IMAGE_1
> o Increase the size of FLASH_AREA_IMAGE_0.
> o Reduce the offset of FLASH_AREA_IMAGE_0.
> 
> * (nrf51xxac.ld)
> o Increase the size of the FLASH region.
> o Set _imghdr_size to 0.
> 
> You will probably also need to move the reboot log flash area.
> Currently, most BSPs place it right between the boot loader and image
> slot 0.
> 
> Let me know if you have any questions.
> 
> Thanks,
> Chris

> #
> # Licensed to the Apache Software Foundation (ASF) under one
> # or more contributor license agreements.  See the NOTICE file
> # distributed with this work for additional information
> # regarding copyright ownership.  The ASF licenses this file
> # to you under the Apache License, Version 2.0 (the
> # "License"); you may not use this file except in compliance
> # with the License.  You may obtain a copy of the License at
> #
> #  http://www.apache.org/licenses/LICENSE-2.0
> #
> # Unless required by applicable law or agreed to in writing,
> # software distributed under the License is distributed on an
> # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> # KIND, either express or implied.  See the License for the
> # specific language governing permissions and limitations
> # under the License.
> #
> 
> bsp.arch: cortex_m0
> bsp.compiler: compiler/arm-none-eabi-m0
> bsp.linkerscript:
> - "hw/bsp/nrf51dk/nrf51xxac.ld"
> - "hw/mcu/nordic/nrf51xxx/nrf51.ld"
> bsp.linkerscript.BOOT_LOADER.OVERWRITE:
>   

Re: Removing or shrinking flash areas

[Christopher Collins]

Hi Jacob,

On Fri, Dec 30, 2016 at 03:28:08PM -0700, Jacob Rosenthal wrote:
> I think deps are a part of the story, but yeah the goal is to successfully
> get a target bsp back to 1 larger flash image, or as many as the common set
> of deps require but shrunk way down.

So you just want to eliminate the need for a second image slot in the
flash map?

There are four flash areas that are relevant here.  Just to be clear: a
flash area is simply a named region of flash.  Each BSP's bsp.yml file
defines its flash map as a sequence of flash areas.

Anyway, the four relevant flash areas are:
* FLASH_AREA_BOOTLOADER:
* FLASH_AREA_IMAGE_0:
* FLASH_AREA_IMAGE_1:
* FLASH_AREA_IMAGE_SCRATCH:

The boot loader area is placed at the board's start address in flash.
That is, the device boots directly into the boot loader.  The boot
loader then possibly swaps the two image slots and jumps to the image in
slot 0.  This process is described in some detail in
boot/bootutil/design.txt.

If your device only needs a single image, you have two options:
1. slot-1 stub
2. no boot loader

Option 1 is the simplest, but option 2 gives you more flash space.  I
will describe both options below.

*** Slot-1 stub:

This option involves greatly reducing the size of the FLASH_AREA_IMAGE_1
area.  When I did this with the nRF51dk, I used an area size of 2kB.  I
think you shld be able to reduce it to the size of an individual
sector (1kB), but I have not tried this.

There are two changes you need to make:
* (bsp.yml)
o Increase the size of FLASH_AREA_IMAGE_0.
o Reduce the size of FLASH_AREA_IMAGE_1.
o Increase the offset of FLASH_AREA_IMAGE_0.

* (nrf51xxac.ld)
o Increase the size of the FLASH region.

I have attached modified versions of these two files that you can try.

*** No boot loader:

This option involves removing the boot loader entirely.
Instead of a boot loader, the image binary is placed at the board's
start address in flash such that the device boots directly into it.
Without a boot loader, you can eliminate the following two areas from
the flash map:
* FLASH_AREA_BOOTLOADER:
* FLASH_AREA_IMAGE_1:

Unfortunately, the scratch area cannot be eliminated at the moment,
since it is required by the reboot log.  I think this is something we
should address soon, but in the meantime, you are stuck with the scratch
area.

It has been a while since I've tried running an image without a boot
loader, so my memory might be a bit foggy.  I think the following steps
are all you need:
* (bsp.yml)
o Remove FLASH_AREA_BOOTLOADER
o Remove FLASH_AREA_IMAGE_1
o Increase the size of FLASH_AREA_IMAGE_0.
o Reduce the offset of FLASH_AREA_IMAGE_0.

* (nrf51xxac.ld)
o Increase the size of the FLASH region.
o Set _imghdr_size to 0.

You will probably also need to move the reboot log flash area.
Currently, most BSPs place it right between the boot loader and image
slot 0.

Let me know if you have any questions.

Thanks,
Chris
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#  http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

bsp.arch: cortex_m0
bsp.compiler: compiler/arm-none-eabi-m0
bsp.linkerscript:
- "hw/bsp/nrf51dk/nrf51xxac.ld"
- "hw/mcu/nordic/nrf51xxx/nrf51.ld"
bsp.linkerscript.BOOT_LOADER.OVERWRITE:
- "hw/bsp/nrf51dk/boot-nrf51xxac.ld"
- "hw/mcu/nordic/nrf51xxx/nrf51.ld"
bsp.part2linkerscript: "hw/bsp/nrf51dk/split-nrf51dk.ld"
bsp.downloadscript: "hw/bsp/nrf51dk/nrf51dk_download.sh"
bsp.debugscript: "hw/bsp/nrf51dk/nrf51dk_debug.sh"

bsp.flash_map:
areas:
# System areas.
FLASH_AREA_BOOTLOADER:
device: 0
offset: 0x
size: 16kB
FLASH_AREA_IMAGE_0:
device: 0
offset: 0x8000
size: 218kB
FLASH_AREA_IMAGE_1:
device: 0
offset: 0x0003e800
size: 2kB
FLASH_AREA_IMAGE_SCRATCH:
device: 0
offset: 0x0003f000
size: 2kB

# User areas.
FLASH_AREA_REBOOT_LOG:
user_id: 0
device: 0
offset: 0x4000
size: 16kB
FLASH_AREA_NFFS:
user_id: 1
device: 0