Re: [VOTE] Graduation of podling Apache Mynewt to Top Level Project

[Kevin Townsend]

+1


On 26/05/17 02:55, aditi hilbert wrote:

Hi all,

We feel confident that we pass all the requirements to graduate.  Below is the 
proposed Graduation Resolution (based on the standard template). A separate 
thread of DISCUSS is also open for this.

The full text of the proposed resolution is as follows:


The incubating Apache Mynewt community believes it is time to graduate to TLP.

Apache Mynewt entered incubation in October of 2015.  Since then, we've 
overcome technical challenges and built Apache's first Operating System for IoT 
devices, and made 6 releases.  Our most recent releases include NimBLE, OIC1.1 
support, MCUbootloader. We are a very helpful and engaged community, ready to 
answer all questions and feedback directed to us via the user list.  We've 
added several committers from different organizations, and are actively 
pursuing others. While we continue working on maturity, all projects are 
ongoing processes, and we believe we no longer need the incubator to continue.

To inform the discussion, here is some basic project information:

Project status:
   http://incubator.apache.org/projects/Mynewt.html

Project website:
   http://Mynewt.incubator.apache.org/

Project documentation:
  https://cwiki.apache.org/confluence/display/MYNEWT/Apache+Mynewt+Project

Maturity assessment:
https://cwiki.apache.org/confluence/display/MYNEWT/Maturity+model

DRAFT of the board resolution is at the bottom of this email

Proposed PMC size: 20 members

Total number of committers: 20 members

Various PMC/Committer affiliations (* indicated chair)
Runtime (6)
Google (1)
Imagination Technologies (1)
Codecoup (4)
Adafruit (1)

4,565 commits on develop
280 closed PR”s on GitHub
39 contributors across all branches
92 forks on Github

dev list averaged ~200 msgs/month in the last one year

647 issues created
465 issues resolved/closed
  
--

Resolution:

Establish the Apache Mynewt Project

WHEREAS, the Board of Directors deems it to be in the best
interests of the Foundation and consistent with the
Foundation's purpose to establish a Project Management
Committee charged with the creation and maintenance of
open-source software, for distribution at no charge to the
public, related to a data management platform that provides
real-time, consistent access to data-intensive applications
throughout widely distributed cloud architectures.

NOW, THEREFORE, BE IT RESOLVED, that a Project Management
Committee (PMC), to be known as the "Apache Mynewt Project",
be and hereby is established pursuant to Bylaws of the
Foundation; and be it further

RESOLVED, that the Apache Mynewt Project be and hereby is
responsible for the creation and maintenance of software
related to an embedded OS optimized for networking and built for remote 
management of constrained devices that are incapable of running either Linux or 
Android.

RESOLVED, that the office of "Vice President, Apache Mynewt" be
and hereby is created, the person holding such office to
serve at the direction of the Board of Directors as the chair
of the Apache Mynewt Project, and to have primary responsibility
for management of the projects within the scope of
responsibility of the Apache Mynewt Project; and be it further

RESOLVED, that the persons listed immediately below be and
hereby are appointed to serve as the initial members of the
Apache Mynewt Project:
*Justin Mclean” <jmcl...@apache.org>
*P. Taylor Goetz” <ptgo...@apache.org>,
*Greg Stein <gst...@apache.org>,
*Jim Jagielski <j...@apache.org>,
*Sterling Hughes <sterl...@apache.org>,
*Marko Kiiskila <ma...@apache.org>,
*will sanfilippo <w...@apache.org>,
Christopher Collins <ccoll...@apache.org>,
Vipul Rahane <vipulrah...@apache.org>,
Fabio Utzig <ut...@apache.org>,
Andrzej Kaczmarek <a...@apache.org>,
Michał Narajowski <na...@apache.org>,
Szymon Janc <j...@apache.org>,
Łukasz Rymanowski <ry...@apache.org>,
Neel Natu <n...@apache.org>,
Peter Snyder <pete...@apache.org>,
Paul Dietrich <paulfdietr...@apache.org,
Julian Ingram <jul...@apache.org>,
Kevin Townsend <kt...@apache.org>,
Aditi Hilbert <ad...@apache.org>

NOW, THEREFORE, BE IT FURTHER RESOLVED, that Justin Mcclean
be appointed to the office of Vice President, Apache Mynewt, to
serve in accordance with and subject to the direction of the
Board of Directors and the Bylaws of the Foundation until
death, resignation, retirement, removal or disqualification,
or until a successor is appointed; and be it further

RESOLVED, that the initial Apache Mynewt PMC be and hereby is
tasked with the creation of a set of bylaws intended to
encourage open development and increased participation in the
Apache Mynewt Project; and be it further

RESOLVED, that the Apache Mynewt Project be and hereby
is tasked with the migration and rationalization of the Apache
Incubator Mynewt pod

Re: iOS newtmgr app evaluation

[Kevin Townsend]

Perhaps it's a limitation of the group mailing list ... no images here 
either, but not a huge problem.


For the firmware uploading process, what I don't like today is the very 
manual process that requires some previous knowledge. You first have to 
upload your image, then reset, then select the new image and click 
'confirm' (or revert) and then reset again. You will also see an error 
when confirming since the return code is not 0 but it's still a 
non-error I believe, though it's been a while. I just think this could 
be easier for customers (one click updates ideally), but I'm afraid the 
entire update process and the various edge cases around it (failed 
transfer, failed confirm, etc.) were a bit hazy, and this underwent some 
fairly active development just before the 1.0.0 release so it was hard 
to keep track of the changes 100% of the time.


The more eyes that look at it the better an eventual release will be, 
though, so thanks for testing!


Kevin


On 06/05/17 04:37, Jacob Rosenthal wrote:

Ill patch my upstream again and try some uploading soon.

One more shot, images attached instead of inlined..

On Fri, May 5, 2017 at 7:24 PM, Kevin Townsend <ke...@adafruit.com 
<mailto:ke...@adafruit.com>> wrote:


Hi Jacob,

The images didn't seem to have made it through? Any feedback would
be appreciated if you see some things to improve, though ... we
were really hoping for some concrete feedback especially on the
firmware update process, but at worst we'll have to publish it as
is and work any issues or improvements out in a future update.

Thanks for taking the time to look, though, and glad you found the
stats useful. I think being able to stream live updates of the
stats is a very useful non-intrusive wireless debug tool.

Kevin


On 06/05/17 03:40, Jacob Rosenthal wrote:

Ill have to steal views like these for the web ble app.
For anyone not in yet

Inline image 1Inline image 2Inline image 3

On Fri, May 5, 2017 at 6:32 PM, Jacob Rosenthal
<jakerosent...@gmail.com <mailto:jakerosent...@gmail.com>
<mailto:jakerosent...@gmail.com
<mailto:jakerosent...@gmail.com>>> wrote:

Wow. What an Amazing dashboard view. I didnt even know all
these
stats were in there. Thanks!

On Wed, Apr 26, 2017 at 11:15 AM, aditi hilbert
<ad...@runtime.io <mailto:ad...@runtime.io>
<mailto:ad...@runtime.io <mailto:ad...@runtime.io>>> wrote:

Hi Kevin,

I am very interested in evaluating this. I do not have any
Swift experience but I am always working on different
demos
for various events and would love to test this out.
My Apple ID/email is aditi.hilb...@gmail.com
<mailto:aditi.hilb...@gmail.com>
<mailto:aditi.hilb...@gmail.com
<mailto:aditi.hilb...@gmail.com>>
<mailto:aditi.hilb...@gmail.com
<mailto:aditi.hilb...@gmail.com>
<mailto:aditi.hilb...@gmail.com
<mailto:aditi.hilb...@gmail.com>>>.

thanks,
aditi


> On Apr 25, 2017, at 11:25 AM, Kevin Townsend
<ke...@adafruit.com <mailto:ke...@adafruit.com>
<mailto:ke...@adafruit.com <mailto:ke...@adafruit.com>>> wrote:
>
> Hi,
>
> We're working on a basic iOS (and later Android) app
to do
some of the tasks newtmgr does, but via a mobile app
you can
download in the app store. It's intentionally limited
for the
moment, but you can monitor stats, tasks and perform
firmware
updates in a common dual bank setup. The most useful
feature
in my opinion is that you can live 'stream' the stats,
so you
have a wireless window in the current state of the HW.
>
> We'd like to have a couple testers for it before we
release,
and I can add a few people to TestFlight for feedback,
but I'd
really like to limit it to people who have the time to
provide
some feedback. The code will be open source once
published,
and we will contribute this to the community, but to
get it
into the app store we need to kick the tires in first.
Ideally, community involved in expanding the app would be
welcome, but we've made a first pass at putting something
together.
>
> If you have the time and interest in evaluating
this, and

Re: nrf52dk hal_bsp_init() and sensor device creation

[Kevin Townsend]

This is my fault most likely ... the goal at the driver level was indeed 
to init with a known default config, but obviously if people think a 
different solution is better there's no reason not to change it!


K.


On 05/05/17 20:39, Vipul Rahane wrote:

I do not know the reason why it is done the way it is but I think you are 
right, the reason might have been to initialize with a default config.



On May 4, 2017, at 3:45 PM, Jacob Rosenthal  wrote:

On Thu, May 4, 2017 at 3:39 PM, Vipul Rahane  wrote:


I have been thinking about calling the driver initialization function
directly from the bsp


Can you comment on why you're not yet init in the BSP? Im thinking power
issues, which is another conversation I started in another thread. Maybe if
the goal was to init with a config that was as low power as possible maybe
its ok to do so in the BSP?


May I know what do you mean by the following ? Isn’t it currently that way.


Shouldnt these BSPs have dependencies on the drivers either way, if only
when the _PRESENT flag is on

Regards,
Vipul Rahane

Re: iOS newtmgr app evaluation

[Kevin Townsend]

You should both have access to the app now.

The important thing is that you are also on the following tags for the 
core repo and utilities:


1. newt core is *mynewt_1_0_0_tag*
2. newt tool  is* mynewt_1_0_0_tag*

Inside the app, the key useful feature is the 'play' icon in the top 
right hand corner. Clicking that enables streaming of stats. You can 
also click it in a specific stats group to see BLE stats tick live, etc.


Please let us know any feedback, ideally with screenshots and detailed 
info, etc.!


I'm limited to how many hours we can pour into this since it's done as 
paid work via an external individual, but we'd like to contribute to the 
project and this seems like a good way.


Best regards,
Kevin

On 25/04/17 20:35, David Simmons wrote:

I'd love to test, and help with development.

santa...@mac.com

Best regards,
dg
--
I'll take credit For the funny typos, the rest I blame on the iPhone spell 
wrecker.


On Apr 25, 2017, at 2:25 PM, Kevin Townsend <ke...@adafruit.com> wrote:

Hi,

We're working on a basic iOS (and later Android) app to do some of the tasks 
newtmgr does, but via a mobile app you can download in the app store. It's 
intentionally limited for the moment, but you can monitor stats, tasks and 
perform firmware updates in a common dual bank setup. The most useful feature 
in my opinion is that you can live 'stream' the stats, so you have a wireless 
window in the current state of the HW.

We'd like to have a couple testers for it before we release, and I can add a 
few people to TestFlight for feedback, but I'd really like to limit it to 
people who have the time to provide some feedback. The code will be open source 
once published, and we will contribute this to the community, but to get it 
into the app store we need to kick the tires in first. Ideally, community 
involved in expanding the app would be welcome, but we've made a first pass at 
putting something together.

If you have the time and interest in evaluating this, and some Swift experience 
would be even better, please reply here or email me the address associated with 
your Apple account and we can add you.

Kevin

PS: Everything is currently based on the 1.0.0 tag to keep things consistent 
across users.

Re: Console improvements - newtmgr and OIC over shell

[Kevin Townsend]

Shell and newtmgr over serial is a requirement for us with the Mynewt 
board were are planning to release soon. There is a USB Serial bridge 
chip on there that we use for both the bootloader and newtmgr commands, 
and I'd certainly like to keep shell support.


K.


On 18/04/17 20:54, Vipul Rahane wrote:

Ok with that as well. However, considering Chris's comments all STM BSPs would 
need a second UART as it is not created in the BSPs.

Regards,
Vipul Rahane


On Apr 18, 2017, at 9:41 AM, Christopher Collins  wrote:

I'm also OK with that.

FYI- apps which need to support both the shell and newtmgr over serial
would need the following modifications:
1. Replace mgmt/newtmgr/transport/nmgr_shell dependency with
   mgmt/newtmgr/transport/nmgr_uart
2. Enable a second UART in the project's syscfg (this would be a BSP
   setting, likely UART_1).  For nRF BSPs, this would be the bit
   banger UART.
3. Specify the appropriate UART as the value of the NMGR_UART syscfg
   setting.  Currently, the console UART is hardcoded per-BSP and
   not configurable via syscfg, but the newtmgr UART is configurable
   (is there a reason this is hardcoded, or is it just a historical
   thing?).

I think that is everything that would need to be done.

Chris

On Tue, Apr 18, 2017 at 09:31:59AM -0700, marko kiiskila wrote:

+1, I’d be ok dropping that support.


On Apr 18, 2017, at 3:58 AM, Michał Narajowski  
wrote:

Hi,

I implemented some console improvements and submitted a pull request
here: https://github.com/apache/incubator-mynewt-core/pull/230

Marko pointed out that it breaks newtmgr and OIC over shell. Should I
fix those issues or can we drop support for that? How many people care
about it?

Best regards,
Michał

Re: Sim Sensor Update

[Kevin Townsend]

There are often situations where you want or need to simulate sensors as
well, for instance if the actual HW isn't available yet or in order to
reproduce a specific issue that can't be easily reproduced in the real
world safely or reliably.

Easy sensor simulation adds a lot of value to mynewt in my opinion, and
having bare minimum examples of how to do this should help people get
started.

Kevin

On Fri, 14 Apr 2017 at 20:22, Vipul Rahane  wrote:

> As I said, IMO it’s for experimentation. It shows you how a sensor device
> is created in the bsp and opened, configured in your app. You can't do
> everything with this as much as you could do with a hardware sensor.
>
> Regards,
> Vipul Rahane
>
> > On Apr 14, 2017, at 9:16 AM, Jacob Rosenthal 
> wrote:
> >
> > So what are you using simaccel0 for?
> >
> > On Thu, Apr 13, 2017 at 11:58 PM, Vipul Rahane  wrote:
> >
> >> d return dummy data which won’t really let you experiment everything but
> >> at the same time let you develop an app which returns some dummy data.
>
>

Re: newt debug .img or elf?

[Kevin Townsend]

Ack, sorry. For posterity sake:

$ newt run bleuart 0.1.0

'newt debug *' works with the elf, I forgot about the version 
requirement with 'run' to also generate the updated .img file, thanks.


K.

On 29/03/17 17:09, Sterling Hughes wrote:

When running on a physical device you need the image version to load, for 
development we just use 0.0.0

Sterling


On Mar 29, 2017, at 7:45 AM, Kevin Townsend <ke...@adafruit.com> wrote:

Was the build process recently updated to output .elf instead of .img?

I'm on 1.0.0 for everything (tools and core repo).

On OS X with GCC 5.4.1 (the latest release from ARM) I get:

*$ newt run bleuart*
Compiling bin/targets/bleuart/generated/src/bleuart-sysinit-app.c
Archiving bleuart-sysinit-app.a
Linking 
/Users/ktown/Dropbox/microBuilder/Code/nRF52/Mynewt/Adafruit_Mynewt/bin/targets/bleuart/app/apps/bleuart/bleuart.elf
Loading app image into slot 1
Error: Cannot find file 
/Users/ktown/Dropbox/microBuilder/Code/nRF52/Mynewt/Adafruit_Mynewt/bin/targets/bleuart/app/apps/bleuart/bleuart.img

*$ ls bin/targets/bleuart/app/apps/bleuart/**
*apps/   apps_bleuart.a.cmd  bleuart.elf.bin* bleuart.elf.lst 
manifest.json
apps_bleuart.a  bleuart.elf*bleuart.elf.cmd bleuart.elf.map

*$ arm-none-eabi-gcc --version**
*arm-none-eabi-gcc (GNU Tools for ARM Embedded Processors) 5.4.1 20160919 
(release) [ARM/embedded-5-branch revision 240496]
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

newt debug .img or elf?

[Kevin Townsend]

Was the build process recently updated to output .elf instead of .img?

I'm on 1.0.0 for everything (tools and core repo).

On OS X with GCC 5.4.1 (the latest release from ARM) I get:

*$ newt run bleuart*
Compiling bin/targets/bleuart/generated/src/bleuart-sysinit-app.c
Archiving bleuart-sysinit-app.a
Linking 
/Users/ktown/Dropbox/microBuilder/Code/nRF52/Mynewt/Adafruit_Mynewt/bin/targets/bleuart/app/apps/bleuart/bleuart.elf

Loading app image into slot 1
Error: Cannot find file 
/Users/ktown/Dropbox/microBuilder/Code/nRF52/Mynewt/Adafruit_Mynewt/bin/targets/bleuart/app/apps/bleuart/bleuart.img


*$ ls bin/targets/bleuart/app/apps/bleuart/**
*apps/   apps_bleuart.a.cmd  bleuart.elf.bin* 
bleuart.elf.lst manifest.json

apps_bleuart.a  bleuart.elf*bleuart.elf.cmd bleuart.elf.map

*$ arm-none-eabi-gcc --version**
*arm-none-eabi-gcc (GNU Tools for ARM Embedded Processors) 5.4.1 
20160919 (release) [ARM/embedded-5-branch revision 240496]

Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Re: SensorsAPI merge from sensors branch to develop and support for different sensors

[Kevin Townsend]

Hi Vipul,


Few days ago I merged the sensors branch into develop.

It consists of the following:
- SensorsAPI support
- LSM303DLHC Acc/Mag/Gyro support
- TSL2561 light sensor support

After the merge I worked on the following and committed code for it as well.
- BNO055 9-Axis Absolute orientation sensor support
- TCS34725 color sensor support( MYNEWT-665 
 SensorAPI: Add TCS34725 
Driver and color sensor support)

Thank you Kevin for helping out wherever possible. This is WIP and any feedback 
is welcome.

I wasn't nearly enough help and regret not being able to be more active 
on this, but I was just reading through the latest commits as you wrote 
this. I'll do some tests tomorrow since I need to get our own code up to 
date with the develop branch. These are all good sensors to have in the 
core codebase, though, since they are all quite relevant to BLE.


K.

Re: Graduation-ready?

[Kevin Townsend]

+1 on my part for graduation


On 14/02/17 20:50, P. Taylor Goetz wrote:

+1 I think Mynewt has been one of the easiest podlings I’ve been involved with 
as a mentor.

If they’d like me to stay on I’d be willing to do so as well.

-Taylor



On Feb 13, 2017, at 11:37 PM, Niclas Hedhman  wrote:

Hi, everyone,

I have been following Mynewt in the "shadows" since I ran into Sterling at
ApacheCon in Vancouver.

The Incubator is filling up faster than graduations are occurring and we
are very keen to graduate podlings that seems to be self-sustaining.

Following the successful release not too long ago, I think Apache Mynewt
should become a TLP,

  1. Release went quite smooth,
  2. A lot of positive activity in community, and podling have brought in
new people from that pool.

Since I am pretty sure Justin will remain on the TLP PMC, I don't worry if
some experience is missing here and there, be that with ASF infra, legal or
otherwise.

I urge Apache Mynewt to discuss graduation and hopefully have a resolution
drafted for March Board meeting (approx 1 month).


Cheers
--
Niclas Hedhman, Software Developer
http://polygene.apache.org  - New Energy for Java

Re: [SensorAPI] Few Questions wrt Sensors

[Kevin Townsend]

Hi Vipul,


On 04/02/17 01:08, Vipul Rahane wrote:

Hello All,

I will be taking over SensorAPI stuff from Sterling.

Kevin:
TSL2561 Driver and LSM303 Driver that you have implemented are pretty good and 
I used both of them. I saw the values coming in using the SensorAPI. I would 
also like to understand about sensors from you or anybody else that is more 
experienced on these as I am kind of new to it, I have some experience but not 
a lot to get a generic understanding.
Happy to help where I can on my side. I'm currently on vacation, but can 
look into this more seriously when I'm back next week.

I am analyzing the API currently along with looking at LSM and TSL 
simultaneously to figure out if we need to add something that is missing to the 
API. I had a few questions:

About LSM303:

* Are +-1g values orientation values or is that actual acceleration ?
+/-1g is actually 'gravity' (1g being 1 'unit' of standard Earth 
gravity), and you can determine orientation based on the influence of 
gravity which is being detected by the accelerometer.  If you are 
getting +1g (or ~9.80665m/s2) on the X axis, it means X is pointing 
'straight up' since you are seeing the 'full' effect of gravity on that 
axis. If you are are seeing -1g on X it means the device is 'flipped' 
180° since you are seeing the negative value (meaning X is pointing 
'straight down'). Depending on which axis you see values on, and the 
relationship between them, you can infer basic orientation on two axis 
with only an accelerometer (normally roll and pitch).


If you add a second sensor, typically a magnetometer, you can add a 
third axis for roll/pitch/heading. Adding a third sensor (a gyroscope) 
you can detect full 360° absolute orientation.


So, you are using an accelerometer and measuring acceleration, but in 
the case of orientation you normally aren't measuring 'acceleration' in 
the sense of movement, you're measuring gravity which is detected as 
acceleration since it presents itself as a constant 'pushing' force with 
a known value and direction, even though the device is potentially immobile.


Does that help?

* Should we be separating out acceleration from orientation(maybe in 
get_interface()?)
I think just providing the raw acceleration values is good enough, and 
'orientation' should actually be a higher level set of helper functions 
that read the raw accel/mag/gyro data and act accordingly. A new sensor 
type could also be implemented for this, but it should rely on the lower 
level raw sensor data types I think.


Separating gravity from acceleration is also problematic. For now, I'd 
keep it simple and just send out the sensor data and users can interpret 
that however they want themselves.

* Zero-g level offset (TyOff) is the standard deviation of the actual output 
signal of a sensor from the ideal output signal. Should we be thinking of 
calibration for sensors in the SensorAPI ? I think in this case it comes with 
factory calibrated values which can be used by a developer. But that will 
change once put on a PCB, hence the concern.
For orientation you will ALWAYS needs to calibrate the sensors to get 
meaningful data (especially the mag, but also the gyro for the 
zero-offset level), and this will always have to be done in the final 
enclosure and environment. So yes, we should have functions to store and 
apply calibration coefficients for the accel/mag/gyro data types (plus 
perhaps some other sensors types, but definately those three). I think 
these should be stored in config memory somewhere, but we need helper 
functions to assign and apply them and just default to '0' values.

* Operating mode of Sensors. Low Power mode Vs Normal Mode for Accelerometer 
and Sleep Mode Vs Single Conversion Mode Vs Continuous Conversion Mode. Modes 
should also be a part of our SensorAPI I believe.
I think having built in power mode control at the API level is extremely 
value (and often neglected), but I think you want to keep things 
manageable as well so this might be pushed off to a second version? I do 
think we should keep power mode in mind with any API level decisions 
though. Sterling has some thoughts on this a while back and made an 
initial proposal that I found very sensible.

*  Should the Filter mode selection also be part of the SensorAPI ?
I think this should be a separate API, but also a key part of any system 
using sensor data. The key thing is to support compatible types between 
the sensor and filter APIs. float32 would be my lowest common 
denominator, especially since the Cortex M4F has excellent float32 
performance (13 cycle divide, 3 cycle multiply off the top of my head 
... which is also as good or better than fixed point), although int32_t 
is nice as a fallback. Float32 will be more painful on an M0, but I 
think most orientation systems will probably be implement on an M4F 
which has a lot more processing power per MHz for this kind of work.

* FIFO config and overrun interrupts and their 

Re: Newtmgr over BLE

[Kevin Townsend]

I'm not sure what the implications would be here on the iOS app we wrote 
that is based on the newtmgr protocol and makes use of the current BLE 
service and characteristics, although we could rewrite it if necessary. 
We use the currently defined newtmgr protocol and GATT services/chars to 
get task info and statistic and stream them live onto the mobile device, 
which I think provides a unique window into your device in a 
non-invasive way.


I'm certainly OK with BLE being removed from the newtmgr tool and having 
that supported by a dedicated tool or service based on UART/HCI, but 
just to be sure are there any implications here for the newtmgr protocol 
running on a mynewt app?  It sounds like we should still be OK with the 
app, though, and this is limited to the newtmgr tool itself not the 
newtmgr protocol implementation or the BLE transport in the Mynewt app.


One advantage of taking the HCI approach talking to a Mynewt target over 
UART is that you can add support to any OS, not just systems that 
support BLE, and not having to depend on a specific OS or BlueZ version. 
You need to do some work up front and it would require another nRF5x 
board, but it solves more problems than it creates in my opinion.


Alternatively, you could use the simulator and try to add some hooks 
into CoreBluetooth and BlueZ (so no additional HW required), but that's 
ultimately frustrating and a significant maintenance burden. UART/HCI 
seems like less work both short and long term, and the requirements for 
an additional nRF5x board seems relatively trivial.


Removing the dependency on the current GO lib and moving to HCI/UART and 
a dedicated Mynewt app is a +1 for me and worth doing before 1.0, even 
if it's a major change ... with the caveat that I hope this won't break 
anything with the newtmgr protocol that will cause us to have to rewrite 
the newtmgr tool we wrote and want to release around the same time as 
1.0.0 final. :)



On 24/01/17 23:49, Christopher Collins wrote:

Hello all,

Recently, I have mentioned some planned BLE-related changes to the
newtmgr tool.  I wanted to share some of what I was thinking.  Please
feel free to comment and criticize as needed.

* All BLE code gets removed from the newtmgr tool.  The gatt library is
   also removed.

* A separate tool, blehostd, runs in its own process.  This tool:
 * Is a Mynewt app running in the Mynewt simulator.
 * Contains the NimBLE host and UART BLE transport.
 * Caches information about connected peers (who is connected and how
   their services, characteristics, and descriptors map to attribute
   handles).
 * Communicates with other processes via JSON over a streaming UNIX domain
   socket.
 * Exposes the host API via JSON-encoded requests, responses, and
   events.

* Controller is accessed via UART device (/dev/cu.<...>).  This could be
   a Mynewt device running the blehci app, or any other controller.

The blehostd app solves a few problems for us:
 * Fixes the bugginess and lack of cross platform support in
   newtmgr (caused by the gatt library).
 * Exposes a generic interface into the NimBLE host for software
   other than newtmgr.

Here is how I envision newtmgr using BLE:

1. If an instance of blehostd isn't already running for this
controller [*]:
 a. Open and bind to a socket.
 b. Start an instance of blehostd, passing it the socket filename
and the controller's /dev filename.

2. Newtmgr tells blehostd to connect to the destination device.
 If the device is already connected: blehostd immediately responds,
 indicating the connection handle of the device.
 Else: The response indicates that the connection attempt is in
 progress.  Eventually, blehostd sends an event indicating
 success and a connection handle, or failure.

3. Newtmgr asks blehostd if the peer's newtmgr characteristic has been
discovered yet.
 Yes: blehostd's response indicates the characteristic's
 attribute handle.
 No: newtmgr tells blehostd to discover the newtmgr
 characteristic.  After discovery completes, newtmgr repeats
 this step so that it knows the characteristic's value handle.

4. Newtmgr builds the CBOR newtmgr request, as always.

5. Newtmgr encodes the CBOR newtmgr request as a bluetooth ATT write
command, and tells blehostd to write it to the peer's newtmgr
characteristic.

6. blehostd immediately responds, indicating success or failure.

7. If successful, blehostd sends an event containing to the peer's
notification (corresponding to its newtmgr response).

[*] This might be a bit tricky to get right.  The goal is to not have to
connect and perform service discovery for each newtmgr command.
Ideally, blehostd would stick around until some idle timeout is
exceeded.  Then, newtmgr could just reuse the existing socket and
blehostd instance to immediately send a follow-up 

Re: NimBLE host advertising data API

[Kevin Townsend]

+1 ... raw buffer plus helpers offers the best of both worlds imho.

Le mar. 24 janv. 2017 à 21:40, will sanfilippo  a écrit :

> I am not sure I have any intelligent comments on this, but that has never
> stopped me from commenting in the past, so…
>
>
>
> I think a byte buffer interface is fine as long as you have helper
> functions to create that buffer. Having folks have to figure out how to
> create an advertisement without any helper functions would be a bad idea
> (imho).
>
>
>
> I am not sure I completely understand your example re:Tx Power Level.
> Would this field still get added by the host or would there be a helper
> function that a developer could call to add the Tx Power Level field to the
> advertisement?
>
>
>
>
>
> > On Jan 24, 2017, at 11:45 AM, Christopher Collins 
> wrote:
>
> >
>
> > Hello all,
>
> >
>
> > I've mentioned this before - I really don't care for the advertising
>
> > data API that we ended up with:
>
> >
> http://mynewt.apache.org/latest/network/ble/ble_hs/ble_gap/functions/ble_gap_adv_set_fields/
>
> >
>
> > I think we should change this API now before the 1.0 release.  I was
>
> > wondering what others think.
>
> >
>
> > The current API is high-level and is relatively easy to use, but
>
> > requires a lot of code space and RAM.  I think a function which just
>
> > takes a raw byte buffer (or mbuf) would be much better.  Then, there
>
> > could be a helper function which converts an instance of `struct
>
> > ble_hs_adv_fields` to a raw byte buffer.
>
> >
>
> > A simple peripheral that always advertises the same data shouldn't be
>
> > burdened with the ble_hs_adv_fields API.
>
> >
>
> > There is actually a rationale as to why the API is the way it is today.
>
> > There are a few fields in the advertisement data that the host can be
>
> > configured to fill in automatically:
>
> >* Flags (contains advertising type).
>
> >* TX Power Level
>
> >
>
> > I figured it would be safer if these values got calculated when
>
> > advertising is initiated.  This is impractical if the host were handed a
>
> > byte buffer rather than a series of fields.
>
> >
>
> > Under the new proposal, the application would need to specify the
>
> > correct advertising type when building the byte buffer, and the tx power
>
> > level would be queried before the advertising procedure is actually
>
> > started.  I don't think this will be a problem in practice, and I think
>
> > the benefits in code size and RAM usage outweigh the API burden.
>
> >
>
> > All thoughts welcome.
>
> >
>
> > Thanks,
>
> > Chris
>
>
>
>

Re: sys/stats and sys/log

[Kevin Townsend]

I don't have any issues with the stub approach myself, and it's easy to 
switch back and forth (no more work than changing syscfg.yml)



On 17/01/17 22:07, marko kiiskila wrote:

Hi,

at the moment it is not very easy to get rid of all code
related to logging and/or statistics.
I ran across this when trying to see how small I can
make an image while keeping BLE and OIC.

Therefore, I was going to create stub packages for
sys/stats and sys/log.

Then, within the app you can choose between a stub or
an actual implementation. We have this same model for
picking up implementation of console.

Alternative would be to make syscfg knobs for these.
However, I think I prefer the stub packages, I believe
that will make the code easier to read (less #ifdef's).

What do you guys think?

Re: SPI/I2C on Arduino Primo

[Kevin Townsend]

You can look at the LSM303DLHC driver I implemented here as a starting 
point for I2C ... that should cover the basics like the I2C 
transactions, which is 1/2 the work already: 
https://github.com/apache/incubator-mynewt-core/tree/sensors_branch/hw/drivers/sensors/lsm303dlhc


You have to look in 'sensors_branch' though.

K.


On 17/01/17 17:32, David G. Simmons wrote:


Devs,

Has anyone used the SPI or I2C on Arduino Primo? Anyone have any 
sample code? Kevin Townsend wrote the Adafruit_BME280_Library in C++ 
and this is the part I'm working with right now, so I do have a 
starting point from that end, but from the MyNewt end?


TIA,
dg
--
David G. Simmons
(919) 534-5099
Web <https://davidgs.com> • Blog <https://davidgs.com/davidgs_blog> • 
Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter 
<http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>

/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
 * http://www.gnupg.com/ Secure your email!!!
 * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!

There are only 2 hard things in computer science: Cache invalidation, 
naming things, and off-by-one errors.

Re: air_quality app

[Kevin Townsend]

I think it should be '/include/pkgname/pgkname.h' for the include files. 
Here's an example I have locally with 'tsl2561' as the root folder where 
this command was run:


$ tree -L 3
.
├── include
│   └── tsl2561
│   └── tsl2561.h
├── pkg.yml
├── src
│   ├── tsl2561.c
│   ├── tsl2561_priv.h
│   └── tsl2561_shell.c
└── syscfg.yml

3 directories, 6 files

K.


On 09/01/17 23:08, David G. Simmons wrote:

I'm working through the air_quality tutorial, and am encountering some problems 
that I'm not sure the root of.

I've created the proper files:
ls -R libs/my_driver/senseair/
total 4
0 drwxr-xr-x  3 dsimmons  staff  102 Jan  9 16:42 include
4 -rw-r--r--  1 dsimmons  staff  869 Jan  9 16:51 pkg.yml
0 drwxr-xr-x  3 dsimmons  staff  102 Jan  9 16:44 src

libs/my_driver/senseair//include:
total 4
4 -rw-r--r--  1 dsimmons  staff  904 Jan  9 16:42 senseair.h

libs/my_driver/senseair//src:
total 4
4 -rw-r--r--  1 dsimmons  staff  839 Jan  9 16:43 senseair.c

The driver's pkg.yml file looks right:
DSimmons-Pro:myproj dsimmons$ cat libs/my_driver/senseair/pkg.yml

pkg.name: libs/my_driver/senseair
pkg.deps:
 - "@apache-mynewt-core/hw/hal"


and I've added
  - libs/my_driver/senseair
to the pkg.deps in my app's pkg.yml file.

All that looks right to me. And is in line with the Tutorial.

When I then add  #include "senseair/senseair.h"

to my main.c I get:
Error: main.c:30:31: fatal error: senseair/senseair.h: No such file or directory

I've tried "" and <> and both produce the same error.

What have I missed?

Best regards,
dg
--
David G. Simmons
(919) 534-5099
Web  • Blog  • Linkedin 
 • Twitter  • GitHub 

/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
  * http://www.gnupg.com/  Secure your email!!!
  * Public key available at keyserver.pgp.com 
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!

There are only 2 hard things in computer science: Cache invalidation, naming 
things, and off-by-one errors.

Re: Sensor API Helper Functions

[Kevin Townsend]

Hi Sterling,
All of the above can probably be fit into a package based on sensor 
'orientation', which might be a natural organizational unit?


I think this belongs in the core sensor package itself, or as a 
sub-directory of hw/sensor (i.e. hw/sensor/orientation). Orientation 
is a very common transformation and having a unified API for it makes 
sense to me, unless you think that the various factors that go into 
orientation (e.g. accuracy) vary greatly based upon the type of sensor 
that you’re looking at, and therefore the algorithms themselves will 
vary greatly.
The orientation code is quite generic and I think should apply to almost 
any accel/mag/gyro as is. It gets slightly more complicated with full 
9DOF sensor fusion (adding gyro support in) since there are certain 
assumptions about gyro unit types that get fed into the algorithm (rad/s 
or dps from the gyro, for example), but if these are clearly defined 
it's up to the user to make sure you feed the filter what it expects in 
terms of units and scale.

FILTERS

A second very important helper unit is something that implements 
basic DSP algorithms to filter raw sensor data. This is critical in 
my opinion, but often neglected in open source systems.


My biggest question comes in: what are the standard types that we 
allow to be passed to these filters?  Is it just floats, or do we 
allow you to, for example, pass accelerometer directly.
If there is one standard type for DSP, I think it should be 
single-precision float, but having an int32_t, fixed-point and maybe 
even double option might be nice. I generally write float and int32_t 
based versions of any DSP filters as a minimum. Part of the reason I 
want to discuss this on the dev list, though, is because you need to 
design things like variable types in from the start, and it isn't always 
obvious how to do it well.


This is easier once you start dealing with concrete examples, though, so 
maybe adding orientation in can be done now since that code is very 
independent, and then we can start looking at how you can add an IIR 
filter between the raw data and the orientation transform to get cleaner 
data (simple moving average implementation, etc.). You will quickly run 
into issues of type, and dealing with filter parameters then.

BONUS POINTS: PEDOMETER

Yeah, I think this is useful.  I’m not sure this belongs in the core, 
but perhaps a package that uses Mynewt?  I could be convinced 
otherwise, if folks think it should be OS level.

This is a separate package to me, yes, since it's very application specific.

K.

Re: BLE certification of the Nimble stack

[Kevin Townsend]

This is something I've been curious about myself as well. Nimble is 
still a work in progress, but I've wondered if there were any plans once 
the final 1.0 release is done and initial development is considered closed.


The SD from Nordic has some significant trade-offs in terms of losing 
real time control of the system, and being limited by the design 
decisions made by Nordic (20 bytes per packet), but it does make it much 
easier to sell products since part of the certification process is 
handled by Nordic. Particularly for low volume, low cost products the 
thousands of dollars/Euros saved here can have a big impact on total per 
device cost, and every bit matters in a startup.


What would the financial burden be of getting a 1.0 Nimble through 
certification along with some certified code for a handful of common 
services/characteristics defined by the SIG? You will still always need 
to register your product with the SIG (which costs money), as well as 
getting FCC certification (which will also cost money), but if the stack 
itself is certified that's one less step to take and one less cheque to 
write for Mynewt users.


K.


On 04/01/17 18:22, Klaus Hagen wrote:

Hi,

Is or will released Nimble stack versions be Bluetooth SIG qualified in the
future? Or are companies integrating the Nimble stack supposed to get their
own qualification done?

Br,
Klaus

Re: [SensorAPI] Default Sensor Config

[Kevin Townsend]

This sounds reasonable to me and at least leaves room for sensible 
defaults, or setting any internal variables to match HW defaults.


K.


On 04/01/17 04:13, Sterling Hughes wrote:
It’s an interesting thought, as you point out there are kinda two 
things at work here.


1- A driver needs to be “config’ed” by default, and where the driver 
init is called is not the right place to do that.


2- The driver doesn’t have any user specific data to call, there 
should be a reasonable/sane default, and I don’t think requiring the 
user to call:


driver->config(NULL);

Is particularly good API design.  I think we should probably dictate 
that config() can be called as many times as possible, and have a 
“config” phase of the sensor API, i.e. when the task gets started, or 
a new sensor gets registered, the sensor API calls config(NULL) to set 
default parameters, and then users can call config(data), if they want 
to reconfigure defaults.


On 29 Dec 2016, at 2:29, Kevin Townsend wrote:

One issues I saw working on some sensor drivers (based on the 
simulated accel as a starting point) is that no 'default' config 
settings are currently being set, and it seems possible that a sensor 
might be initialised without calling the config function explicitly.


The config system is great (highly flexible and easy to understand), 
but would it be better to have a default config setting that is set 
when the sensor is initialized, or should running the config function 
at least once be considered mandatory and the assumption made that it 
is always being called? In that case, should we return an error if 
something like a read function is called but the sensor hasn't been 
configured yet to put it into a known state (and probably wake the 
sensor up since many devices will default to booting into power down 
mode)?

Re: MyNewt Mqueue not working

[Kevin Townsend]

I had a similar problem last week trying to understand the callbacks 
with events queues, but perhaps the following sample code I put together 
will clarify something. It uses the second approach mentioned by Chris, 
which is a dedicated processing task for the event queues so that I know 
the priority level the events will be executed within.


This code sets up a HW interrupt on a GPIO pin, and when we enter the 
interrupt service routine we add an event to the eventq. This event is 
handled outside the ISR context at a later less critical time. It just 
does blinky now which could be safely done inside the ISR, of course, 
but it's a useful real world example of when you would want to use event 
queues in my opinion to get in and out of the ISR as quickly as possible.


Kevin

#include "os/os.h"
#include "bsp/bsp.h"
#include "hal/hal_gpio.h"
#include 
#include 
#ifdef ARCH_sim
#include 
#endif

#define IRQ_PIN (20)
#define EVENT_TASK_PRIO (1)
#define EVENT_STACK_SIZEOS_STACK_ALIGN(128)

struct os_task event_task;
os_stack_t event_stack[EVENT_STACK_SIZE];
static struct os_eventq irq_evq;

static int init_tasks(void);
static void gpio_task_irq_deferred_handler(struct os_event *);

static struct os_event gpio_irq_handle_event = {
.ev_cb = gpio_task_irq_deferred_handler,
};

/**
 * This function will be called when the gpio_irq_handle_event is pulled
 * from the message queue.
 */
static void
gpio_task_irq_deferred_handler(struct os_event *ev)
{
hal_gpio_toggle(LED_2);
}

/**
 * This task serves as a container for the shell and newtmgr packages.  
These

 * packages enqueue timer events when they need this task to do work.
 */
static void
event_task_handler(void *arg)
{
while (1) {
os_eventq_run(_evq);
}
}

/**
 * This function handles the HW GPIO interrupt and registers an event in
 * the event queue to defer taking action here in the ISR context.
 */
static void
gpio_irq_handler(void *arg)
{
/* Add item to event queue for processing later, etc. ... */
os_eventq_put(_evq, _irq_handle_event);
}

/**
 * init_tasks
 *
 * Called by main.c after os_init(). This function performs initializations
 * that are required before tasks are running.
 *
 * @return int 0 success; error otherwise.
 */
static int
init_tasks(void)
{
/* Init IRQ pin: falling edge, pullup enabled */
hal_gpio_irq_init(IRQ_PIN,
  gpio_irq_handler,
  NULL,
  HAL_GPIO_TRIG_FALLING,
  HAL_GPIO_PULL_UP);

/* Enable the IRQ */
hal_gpio_irq_enable(IRQ_PIN);

/* Setup the LED pins */
hal_gpio_init_out(LED_BLINK_PIN, 1);
hal_gpio_init_out(LED_2, 0);

os_task_init(_task, "irq", event_task_handler, NULL,
EVENT_TASK_PRIO, OS_WAIT_FOREVER, event_stack, 
EVENT_STACK_SIZE);


os_eventq_init(_evq);
os_eventq_dflt_set(_evq);

return 0;
}

/**
 * main
 *
 * The main function for the project. This function initializes the os, 
calls
 * init_tasks to initialize tasks (and possibly other objects), then 
starts the

 * OS. We should not return from os start.
 *
 * @return int NOTE: this function should never return!
 */
int
main(int argc, char **argv)
{
int rc;

#ifdef ARCH_sim
mcu_sim_parse_args(argc, argv);
#endif

os_init();

rc = init_tasks();
os_start();

/* os start should never return. If it does, this should be an error */
assert(0);

return rc;
}


On 02/01/17 11:26, then yon wrote:

Dear Chris,

I tried to apply your solution but i stuck somewhere in middle as the 
handler parameter is diff that cause build error


void shell_task_handler(void *arg)
static void queue_task_handler(struct os_event *ev)

1. At os_eventq_init(_task_evq);

2. At os_task_init(_task, "queue", queue_task_handler, NULL, 
QUEUE_TASK_PRIO, OS_WAIT_FOREVER, queue_stack, QUEUE_STACK_SIZE);



Thank you for your fast response and happy new year.

Regards,
Then Yoong Ze

On 2/1/2017 2:25 PM, Christopher Collins wrote:

Hi Then,

On Mon, Jan 02, 2017 at 01:23:33PM +0800, then yon wrote:

Dear Support,

I been using MyNewt v1.0.0-dev, and i tried to make use of Mqueue.

What i did was following the tutorial below:

https://mynewt.apache.org/os/core_os/mqueue/mqueue/
Since the tutorial is not updated, i tried to modify it but 
unfortunately i can't make it work.

[...]

Your code looks good.  There is just one crucial step missing: a task
needs to poll the eventq you created (queue_task_evq).  Just to clarify
how mqueues work: when os_mqueue_put() is called, two things happen:
 1. The specified mbuf is placed on the mqueue.
 2. The event associated with the mqueue is put on the specified
eventq, but only if the event is not already enqueued.

If a task is listening on the eventq, then it can your mqueue processing
function when it sees the mqueue event.

I think you have two options:
 1. Remove queue_task_evq from your code; use the other eventq

Monochrome Display OS Device

[Kevin Townsend]

tl;dr: Is there bandwidth and interest to collaborate on stubs for an 
os_dev based mono display driver, monogfx drawing package, and basic 
simulator/test-code, if I'm willing to fill some of the stubs and driver 
details in since I've written a lot of display drivers over the years?


Bloated version follows ...

Has anyone here made any effort to implement mono or RGB displays as a 
generic os_dev?


I've started working on a reference Mynewt app where I will use an 
inexpensive 128x32 pixel OLED display where the top 128x8 pixels are a 
status area controlled by the display task, containing info like the 
RSSI level, battery level, connection status, etc. The bottom 128x24 
pixels are accessible as a buffer that can be update in any other task 
handler to display context-specific information. I've attached a .md 
file with my initial thoughts on the display management task, just as a 
FYI. I'm writing it as a reference of how to handle pass events around 
between tasks, and how to organize tasks by priority level.


I'm happy to contribute a driver for the SSD1306 to Mynewt since the 
OLED displays that are it are affordable, compact, can be reliable 
sourced, and most of all are appropriate for small individual sensor 
nodes due to their size and readability.


This would be much better as an os_dev, though, based on a generic 
monochrome display, with a generic monogfx drawing library that can be 
used with it to handle basic drawing, rendering text, etc.


Monochrome displays should be separate devices from RGB displays, in my 
opinion, because they behave very differently:


Mono displays tend to be buffer based, where you have 1 bit per pixel in 
an appropriately sized buffer, and all drawing happens on that buffer. 
When you want to refresh the display, you read the buffer and dump the 
pixels to the display.


Inexpensive RGB displays tend to have internal memory buffers, and 
variable pixel color widths and structures (RGB16, RGB18, RGB24, 8-bit 
lookup tables). The R/G/B components can also change order depending on 
the display and controller. Different controllers also have different HW 
accelerated functions for drawing, and you need to design in the option 
to take advantage of these accelerated function, OR fallback to a purely 
SW based alternative when no HW option is present (fill screen, draw 
basic shapes, invert the display, etc.). The drawing library would be 
different enough that I think this should be a separate device type with 
a seperate drawing library ('rgbgfx'?)


Monochrome is a better place to start, though, since these are far more 
common in embedded systems, have faster updates, and are less expensive.


Is there any bandwidth at present for other Mynewt participants to scope 
out an os_dev based display and monogfx library following the ADC and 
sensor design pattern?


I can contribute some HW drivers (starting with the SSD1306), some basic 
monogfx drawing code (lines, shapes, text) and a socket-based display 
simulator, but I would prefer the generic os_dev API to be a 
collaborative effort since it would need to be maintained longer term.


For the simulator, this can be either based on writing timestamped image 
files directly to disk every time the display is refreshed, or I can 
make a socket based visual simulator that potentially also includes user 
feedback like simulating buttons or a touch screen, with the ability to 
export screen shots. I think having a simulator is worth the work since 
UI develop is an endless cycle of code, build, test, tweak, repeat, and 
being able to send screen shots around at an early stage where not 
everyone has access to HW is very useful.


If there isn't the bandwidth for this, I'll just put together a 
stand-alone SSD1306 driver and talk directly to that, though, and we can 
come back to an os_dev based version at a later date. It isn't something 
I think I'd want to design (as a dev) in a corner since it likely isn't 
to be useful longer term designed in isolation.


K.

PS: The displays I intend to use are the following, which can be 
reliably sourced and are relatively inexpensive in volume, and 
appropriate for real world use in small sensor nodes:


- https://www.adafruit.com/product/2900
- https://www.adafruit.com/products/938
- https://www.adafruit.com/products/326
- https://www.adafruit.com/products/931

# OLED Display

- **Display Name**: oled
- **Priority**: 200
- **Stack Size**: 64 (256 bytes)

Controls and updates the 128x32 pixel mono OLED display based on incoming
events.

## Overview

This task controls the HW refresh of the 128x32 pixel monochrome OLED display.

It will update the display contents based on a series of events that can be
passed in to the internal message queue, such as updating the displayed RSSI
level, battery level, or updating the 128x24 pixel user portion of the screen.

### Display Sections

The display is divided into two sections:

The **status bar section** is 128x8 pixels. This 

Re: Schedule task with strict fixed timing and variable workload

[Kevin Townsend]

I think you're right that this should follow the ADC model since the 
constraints are similar.



On 28/12/16 19:19, Sterling Hughes wrote:

+1 for PWM support, but as a driver, not HAL.

https://issues.apache.org/jira/browse/MYNEWT-525

Sterling

Re: [nRF52 HAL I2C] Possible 'probe' issue?

[Kevin Townsend]

Hi Greg,

 /* Scan all valid I2C addresses (0x03..0x77) */
 for (addr = 0x08; addr < 0x78; addr++) {

fix the comment :-P

I fixed that locally, yes ... caught it after copying and pasting :)

Well, the address isn't necessarily "invalid", it just isn't defined by the
I2C specification.

Definitely, those reserved addresses should not be probed.

For the API, I don't think an error is warranted, as address 0x00 is
well-defined as a "general call" address. Calling applications should know
what they are doing with the 0-7 addresses, and I don't think mynewt should
hamper them without a strong rationale for doing so. (or possibly a
separate set of APIs for the behaviors of those special addresses)

Fair enough. 0x00 is valid, yes (though not every IC supports general 
call). I'm just surprised using reserved addresses would throw the 
entire peripheral into the weeds, but it's fair to say that the burden 
is on the end user not to use those registers. It just seems like 
something that might be hard to debug if you make a typo, but again ... 
the burden is on you to know what you are sending to the device.


At least the issue is clear now, though, thanks for looking into this 
and highlighting the cause of the problem! I spent a day reading all the 
I2C details from the product spec, checking errata, etc., when the 
problem was right in front of my nose. :P


K.

Re: [nRF52 HAL I2C] Possible 'probe' issue?

[Kevin Townsend]

BTW, an easy workaround for this would just be rejecting any requests 
that aren't in the 0x8..0x77 range on the nRF52. That's valid on any 
chip, to be honest, though only an issue on the nRF52.


I'll see if I spot anything on the logic analyzer tomorrow, though, but 
I think this is internal to the peripheral block itself inside the chip 
and so it won't likely appear just looking at the I2C transactions. But 
I'll let you know once I have it hooked up again.



On 27/12/16 01:51, will sanfilippo wrote:

What I am going to say is obvious, but has anyone hooked up an i2c analyzer to 
see what is actually going on in the two cases? I dont see why the peripheral 
would act differently based on the address itself; something must be happening 
on the bus causing the peripheral to act differently.

When I was messing around with the nrf i2c peripheral I used the probe API but 
only for the address I was expecting.

The way we are programming the nrf peripheral might indeed be an issue here but 
without knowing more about how devices act and more about the i2c protocol 
itself I dont think I can offer anything intelligent here :-)

I dont have an analyzer with me but if no one can get a hold of one and test it 
out, I will try to get something set up later in the week.

Re: [nRF52 HAL I2C] Possible 'probe' issue?

[Kevin Townsend]

I'll hook a logic analyzer up again tommorow with and without the 
invalid address range and see what the difference is, but it seems like 
a single incorrect address usage is sending the I2C block into the weeds 
for some reason. Nordic's SDK has the same behaviour with the SD and 
their drivers.



On 27/12/16 01:51, will sanfilippo wrote:

What I am going to say is obvious, but has anyone hooked up an i2c analyzer to 
see what is actually going on in the two cases? I dont see why the peripheral 
would act differently based on the address itself; something must be happening 
on the bus causing the peripheral to act differently.

When I was messing around with the nrf i2c peripheral I used the probe API but 
only for the address I was expecting.

The way we are programming the nrf peripheral might indeed be an issue here but 
without knowing more about how devices act and more about the i2c protocol 
itself I dont think I can offer anything intelligent here :-)

I dont have an analyzer with me but if no one can get a hold of one and test it 
out, I will try to get something set up later in the week.

Re: [nRF52 HAL I2C] Possible 'probe' issue?

[Kevin Townsend]

And just to highlight the point, here is the results
using 0x03 at the starting point on an identical setup:

   i2cscan
   9116:Scanning I2C bus 0
 0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
   00:  -- -- -- -- -- -- -- -- -- -- -- -- --
   10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   70: -- -- -- -- -- -- -- --
   Found 0 devices on I2C bus 0

The peripheral shouldn't die because of a faulty addr,
though, and it isn't a behaviour I've seen before. :)

That sounds suspiciously like a silicon level problem with
the I2C peripheral itself, though it would need further
testing to confirm.

Re: [nRF52 HAL I2C] Possible 'probe' issue?

[Kevin Townsend]

Hi Greg,

Well, unexpected but the problem is indeed the faulty address
requests at the beginning and correctly setting the start
address to 0x08 causes the LSM303 to respond:

   133728:Scanning I2C bus 0
 0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
   00: -- -- -- -- -- -- -- --
   10: -- -- -- -- -- -- -- -- -- 19 -- -- -- -- 1e --
   20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   70: -- -- -- -- -- -- -- --
   Found 2 devices on I2C bus 0

The code is below:

   static int
   shell_i2cscan_cmd(int argc, char **argv)
   {
uint8_t addr;
int32_t timeout = OS_TICKS_PER_SEC / 10;
uint8_t dev_count = 0;

console_printf("Scanning I2C bus 0\n"
   " 0  1  2  3  4  5  6  7  8  9 a  b  c  d 
   e  f\n"

   "00: ");

/* Scan all valid I2C addresses (0x03..0x77) */
for (addr = 0x08; addr < 0x78; addr++) {
int rc = hal_i2c_master_probe(0, addr, timeout);
/* Print addr header every 16 bytes */
if (!(addr % 16)) {
  console_printf("\n%02x: ", addr);
}
/* Display the addr if a response was received */
if (!rc) {
console_printf("%02x ", addr);
dev_count++;
} else {
console_printf("-- ");
}
}
console_printf("\nFound %u devices on I2C bus 0\n", dev_count);

return 0;
   }

However, this does still point to a design issue with the I2C
peripheral, and other ICs with identical code combined with this
particular sensor don't exhibit the same behaviour.

Perhaps at a minimum we should check the address in the probe
function and return an appropriate error code if the requested
address is < 0x08, or reject /any/ I2C transaction where the
address is invalid on the nRF52?

Kevin


On 27/12/16 00:15, Greg Stein wrote:

Hey Kevin, all,

I've done quite a bit of I2C, and Sterling pointed out this thread for me
to add some thoughts.

First up, the probe should start scanning at 0x08 (rather than 0x03). Per
the I2C spec, Rev 6, Section 3.1.12, the first and last 8 addresses (in
7-bit addressing) are reserved. Thus 0x08 to 0x77.

Second, the fact that more addresses show up is very disconcerting. For
example, given the 0x29 address for the TCS3472x, how does 0x2a show up?
That is *two* bits changing on the bus. I could see noise introducing one
bit... but two?

And lastly, it seems very weird that the STOP bit is at issue. The N/ACK
occurs before the STOP event is sent. I could possibly imagine bus
weirdness, and parts' state machines going off track, but if the bus is run
properly, then those state machines should be working fine. I'd expect way
more problems when you stop sending that STOP event.

I haven't dug into the code. Is the probe using hardware, or bit banging?
And if hardware, what is the master in your scenario? I found the
microcontroller on the original RPi cannot do repeated starts. So masters
*do* have bugs. Hooking up that scope would be great, to see if your master
is properly obeying the I2C protocol. Particularly given the double-bit
ghost appearance of new nodes.

Cheers,
-g


On Thu, Dec 22, 2016 at 4:54 PM, Kevin Townsend <ke...@adafruit.com> wrote:


As a follow up to the unexpected I2C scan issue with the LSM303DLHC, I
looked at the errata for the nRF52832 and it may be related to this issue
described below (since the scan problem is unique to the nRF52). A second
STOP bit may be causing problem with the LSM303DLHC inside the probe
function. I'll have to look on a scope to see whether this is happening or
not. There were no other likely errata issues listed in the latest document.

Source: http://infocenter.nordicsemi.com/pdf/nRF52832_Rev_1_Errata_v
1.3.pdf

*3.22 [83] TWIS: STOPPED event occurs twice if the STOP task is triggered
during a transaction*

This anomaly applies to IC Rev. Rev 1, build codes QFAA-B00, QFAB-B00,
CIAA-B00. It was inherited from the previous IC revision Engineering C.

*Symptoms**
*STOPPED event is set after clearing it.

*Conditions**
*The STOP task is triggered during a transaction.

*Consequences**
*STOPPED event occurs twice: When the STOP task is fired and when the
master issues a stop condition on the bus. This could provoke an extra
interrupt or a failure in the TWIS driver.

*Workaround**
*The last STOPPED event must be accounted for in software.



On 19/12/16 14:37, Kevin Townsend wrote:


While adding a test driver for the new sensor API, we've been trying to
debug an issue where a specific sensor (the LSM303DLHC) doesn't show up in
an 

Re: Schedule task with strict fixed timing and variable workload

[Kevin Townsend]

Hi Will,

Thanks for the feedback.


1) Unless you are the highest priority task, other tasks can run which could 
cause delays, and thus you are not waking up at the desired time.
Yeah, everything is based on the assumption that priority is resolved by 
design, and that the scheduling constraints are realistic and well 
understood in the system.

2) Using os_time_delay() and os_callout_reset() gives you a 1 os time tick 
resolution, so if your ticks are 10 msecs, you will be off by up to 9 msecs. A 
1 msec ticker gets you off by up to 1 msec.

Using a task to post another task is a bit heavyweight; I would use a timer or 
a callout to do this. A timer would solve your “tick resolution” issue but 
would not solve the problem of a task wake up getting delayed.
A timer would be a valid solution as well, yes. I haven't looked 
seriously at the timer HAL yet, but I'll have a look right now and give 
it a try just to familiarize myself with it.


I saw earlier today that there wasn't PWM support and wanted to see if 
that was something that could be easily added to the timer HAL since 
it's a common requirement that should probably be included (motor 
control, dimming control, etc.). Different issue though!

It would not be hard to add an API to os_callout.c that could be used for this. 
Something like “os_callout_reset_at” or “os_callout_reset_tick” which you could 
pass in a specified os tick. Not sure what the API should do if you missed the 
os tick (return -1 and not post the task)? This would be simple code to use; 
just add the sample rate to the last time and call the new os_calllout API.
Personally, I think this would make a lot of sense and solve what is 
bound to be a common problem, and perhaps you can have a flag to define 
the behaviour when you miss the delay. Either you return an error code, 
OR you fire the task as soon as you can (though ideally still with a 
flag of some sort to know you're late), but having the option between 
the two should solve some problems.


K.

Re: Schedule task with strict fixed timing and variable workload

[Kevin Townsend]

Hi Fabio,

Thanks for the feedback and suggestion.

I didn't think to have separate sync and read tasks. It does add a 
decent amount of code when this could be solved with one or two lines 
and a built in helper class, but it's a workable and reliable solution 
today.


I also tested the code below but there are several potential problems 
with it:


   static void
   blinky_task_handler(void *arg)
   {
static os_time_t start;
static os_time_t stop;
hal_gpio_init_out(LED_BLINK_PIN, 1);

while (1) {
/* Measure the starting tick */
start = os_time_get();

/* Toggle the LED */
hal_gpio_toggle(LED_BLINK_PIN);

/* Measure the ending tick */
stop = os_time_get();

/* Check for overflow in tick counter */
if (stop >= start) {
os_time_delay(OS_TICKS_PER_SEC/BLINKY_RATE_HZ - (stop -
   start));
} else {
/* Overflow occured */
os_time_delay(OS_TICKS_PER_SEC/BLINKY_RATE_HZ - stop +
(UINT_MAX - start) );
}
}
   }

This doesn't account for situations where the read event might run 
/over/ the desired rate either, which will require a check before the 
overflow test if 'stop-start > OS_TICKS_PER_SECOND/BLINKY_RATE_HZ', at 
which point you probably want to flag the timing overrun and maybe get a 
new sample right away. There are likely other issues I'm not seeing as 
well, such as making sure we don't exceed 1/2 the OS time epoch in our 
delay.


Having a single helper function would let you encapsulate all these edge 
cases in one call might be useful, but perhaps someone has a suggestion 
about the naming or some other helpers that are worth considering ... or 
disagrees that this should be included in the core at all!?


FreeRTOS, for example, has vTaskDelayUntil(): 
http://www.freertos.org/vtaskdelayuntil.html


K.

Re: Fwd: Fwd: BLE quickly disconnects after connect on iPhone (LightBlue)

[Kevin Townsend]

Hi Chris,

For option 2, you should modify your target's system configuration.  You
can increase the mbuf count to 16 as follows:

 newt target syscfg set  syscfg=MSYS_1_BLOCK_COUNT=16

Was the 'target syscfg set' support recently added? I don't see this in 
the help menu for newt based on the latest version in 'develop'?


If this is possible from the command line I wanted to add a note on it 
to our internal documentation, but I must be on the wrong branch.


Kevin

Sensor API Helper Functions

[Kevin Townsend]

I'm probably getting ahead of myself diving into this before the sensor 
API is nailed down, but if we wanted to add some generic helper 
functions that make use of the sensor API, what would be a good location 
or what to organize them as a package? It isn't always easy to organize 
things like this by sensor type since there is a lot of overlap.


I can think of two helper packages we might want to add as a core test 
case, that are useful for testing and getting actionable, useful output 
from the sensor API:


ORIENTATION

Using only an accelerometer you can get rough roll/pitch values: 
https://github.com/microbuilder/LPC11U_LPC13U_CodeBase/blob/development/src/drivers/sensors/accelerometers/accelerometers.c#L81


If you add a magnetometer you can get rough roll/pitch/heading values: 
https://github.com/microbuilder/LPC11U_LPC13U_CodeBase/blob/development/src/drivers/sensors/sensorfusion.c#L78


If you add a gyroscope, you can do proper 9DOF sensor fusion using a 
variety of algorithms and get better quality roll/pitch heading 
(madgwick, mahoney, etc.).


All of this will require at a minimum magnetometer calibration to 
compensate for hard and soft iron errors, and probably gyroscope 
calibration to remove the zero drift error. Accel calibration helps, but 
isn't as critical as the mag and gyro.


All of the above can probably be fit into a package based on sensor 
'orientation', which might be a natural organizational unit?


FILTERS

A second very important helper unit is something that implements basic 
DSP algorithms to filter raw sensor data. This is critical in my 
opinion, but often neglected in open source systems.


This should contain things like a simple moving average filter, a basic 
FIR and IIR filter (that can take coefficients from Octave/Matlab), and 
a other very common but useful filters that will help remove noise from 
less expensive sensors like the lsm303dlhc.


For example, see:

 * *iir.c* here:
   
https://github.com/microbuilder/LPC11U_LPC13U_CodeBase/tree/development/src/drivers/filters/iir
 * Simple and weighted moving average filters here:
   
https://github.com/microbuilder/LPC11U_LPC13U_CodeBase/tree/development/src/drivers/filters/ma

The iir.c example above has some notes on calculating the coefficients 
for a 2 pole butterworth filter, or determining the coefficients 
internally. It's a limited example, but I think the right direction to 
move things:


 * Octave Butterworth Example:
   
https://github.com/microbuilder/LPC11U_LPC13U_CodeBase/blob/development/src/drivers/filters/iir/iir.c#L57
 * Calculating Butterworth Directly:
   
https://github.com/microbuilder/LPC11U_LPC13U_CodeBase/blob/development/src/drivers/filters/iir/iir.c#L269

BONUS POINTS: PEDOMETER

A third package that could be implemented that would be a useful example 
for BLE is taking accelerometer data and turning it into a pedometer, 
which can be done with some reasonably straight forward code. I have 
some code I can recycle for this, but it isn't publicly available on 
Github.


I think this is interesting for BLE and I can help here, but the two 
above are more critical.


Kevin

PS: Some of the code above is quite old and I'm not terribly proud of it 
and would do things differently today and I've reimplemented these 
elsewhere differently, but I think the examples are at least valid as an 
illustration or where things need to move to be useful in the real world.

Re: [nRF52 HAL I2C] Possible 'probe' issue?

[Kevin Townsend]

As a follow up to the unexpected I2C scan issue with the LSM303DLHC, I 
looked at the errata for the nRF52832 and it may be related to this 
issue described below (since the scan problem is unique to the nRF52). A 
second STOP bit may be causing problem with the LSM303DLHC inside the 
probe function. I'll have to look on a scope to see whether this is 
happening or not. There were no other likely errata issues listed in the 
latest document.


Source: http://infocenter.nordicsemi.com/pdf/nRF52832_Rev_1_Errata_v1.3.pdf

*3.22 [83] TWIS: STOPPED event occurs twice if the STOP task is 
triggered during a transaction*


This anomaly applies to IC Rev. Rev 1, build codes QFAA-B00, QFAB-B00, 
CIAA-B00. It was inherited from the previous IC revision Engineering C.


*Symptoms**
*STOPPED event is set after clearing it.

*Conditions**
*The STOP task is triggered during a transaction.

*Consequences**
*STOPPED event occurs twice: When the STOP task is fired and when the 
master issues a stop condition on the bus. This could provoke an extra 
interrupt or a failure in the TWIS driver.


*Workaround**
*The last STOPPED event must be accounted for in software.


On 19/12/16 14:37, Kevin Townsend wrote:
While adding a test driver for the new sensor API, we've been trying 
to debug an issue where a specific sensor (the LSM303DLHC) doesn't 
show up in an I2C bus scan (using the 'probe' command in the I2C HAL), 
using the following shell command on the nRF52, but it DOES work on 
other targets:


// Command handler prototype declaration
static int shell_i2cscan_cmd(int argc, char **argv);

// Shell command struct
static struct shell_cmd shell_i2cscan_cmd_struct = {
.sc_cmd = "i2cscan",
.sc_cmd_func = shell_i2cscan_cmd
};

// i2cscan command handler
static int
shell_i2cscan_cmd(int argc, char **argv)
{
uint8_t addr;
int32_t timeout = OS_TICKS_PER_SEC / 10;
uint8_t dev_count = 0;

console_printf("Scanning I2C bus 0\n"
   " 0  1  2  3  4  5  6  7  8  9  a  b c  d 
e  f\n"

   "00:  ");

/* Scan all valid I2C addresses (0x03..0x77) */
for (addr = 0x03; addr < 0x78; addr++) {
int rc = hal_i2c_master_probe(0, addr, timeout);
/* Print addr header every 16 bytes */
if (!(addr % 16)) {
  console_printf("\n%02x: ", addr);
}
/* Display the addr if a response was received */
if (!rc) {
console_printf("%02x ", addr);
dev_count++;
} else {
console_printf("-- ");
}
}
console_printf("\nFound %u devices on I2C bus 0\n", dev_count);

return 0;
}

The HW (https://www.adafruit.com/product/1120 or 
https://www.adafruit.com/products/1714) has been used on about 20 
different MCUs by me, and it's usually one of the first drivers I 
write so I'm confident the HW is good.


What is unusual is that the LSM303DLHC is the only sensor that doesn't 
show up. We have 5 other I2C sensors connected to the test system that 
show up fine.


I thought it might be a HW issue we haven't seen before since we have 
5V/3V level shifting, but checking the LSM303DLHC datasheet I'm quite 
confident the HW breakout is fine and we are using the recommended 
values (10K pullups on SCL/SDA, etc.). It fails at both 100kHz and 400kHz.


I think the issue is ultimately here in the probe function with the 
STOP bit that is being sent (which I think is the right default). 
Changing it to not send the STOP condition does bring more devices up 
on the I2C bus:


int
hal_i2c_master_probe(uint8_t i2c_num, uint8_t address, uint32_t timo)
{
struct hal_i2c_master_data rx;
uint8_t buf;

rx.address = address;
rx.buffer = 
rx.len = 1;

*return hal_i2c_master_read(i2c_num, , timo, 0);*
}

With the following sensors attached:

LSM303DLHC Accel: 0x19
LSM303DLHC Mag:   0x1E
TCS3472x: 0x29
FXAS21002C:   0x21
FXOS8700CQ:   0x1F
BMP280:   0x77
Si7201:   0x40

I get this with the default behaviour of the STOP bit being enabled 
... everything show up except the LS303DLHC accel/mag:


i2cscan
12275:Scanning I2C bus 0
 0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:  -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 1f
20: -- 21 -- -- -- -- -- -- -- 29 -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: 40 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- 77
Found 5 devices on I2C bus 0


When I disable the STOP (lastop=0) bit I get more add

LSM303DLHC Sensor Driver

[Kevin Townsend]

I made a first pass at a driver for the LSM303DLHC using the new sensor 
API. This sensor was chosen since it has multiple sensors in a single IC 
so it can be used to test the multiple data type functionality in the 
sensor API. The magnetometer and accelerometer can also be combined 
later to generate basic orientation data to test BG processing of sensor 
data streams with useful output over the air. A simple step detection 
algorithm can also be implemented based solely on streaming accel data 
to an appropriate algorithm.


The pull request is available here along with some basic notes on using 
it: https://github.com/apache/incubator-mynewt-core/pull/147


Feel free to modify anything, this is just an attempt to get things 
moving with real HW (although having a simulated accelerometer is also 
very useful for testing purposes).


The current driver implements statistics and logging (configurable via 
syscfg.yml flags). For statistics, the sensor reads are tracked for each 
range settings for the accelerometer since the final driver will 
probably implement an auto ranging system, and it is helpful to evaluate 
the ranging system via statistics.


The magnetometer and temperature sensor will be added as the sensor API 
stabilizes, but these are easy additions.


Kevin

[nRF52 HAL I2C] Possible 'probe' issue?

[Kevin Townsend]

While adding a test driver for the new sensor API, we've been trying to 
debug an issue where a specific sensor (the LSM303DLHC) doesn't show up 
in an I2C bus scan (using the 'probe' command in the I2C HAL), using the 
following shell command on the nRF52, but it DOES work on other targets:


   // Command handler prototype declaration
   static int shell_i2cscan_cmd(int argc, char **argv);

   // Shell command struct
   static struct shell_cmd shell_i2cscan_cmd_struct = {
.sc_cmd = "i2cscan",
.sc_cmd_func = shell_i2cscan_cmd
   };

   // i2cscan command handler
   static int
   shell_i2cscan_cmd(int argc, char **argv)
   {
uint8_t addr;
int32_t timeout = OS_TICKS_PER_SEC / 10;
uint8_t dev_count = 0;

console_printf("Scanning I2C bus 0\n"
   " 0  1  2  3  4  5  6  7  8  9  a  b c  d 
   e  f\n"

   "00:  ");

/* Scan all valid I2C addresses (0x03..0x77) */
for (addr = 0x03; addr < 0x78; addr++) {
int rc = hal_i2c_master_probe(0, addr, timeout);
/* Print addr header every 16 bytes */
if (!(addr % 16)) {
  console_printf("\n%02x: ", addr);
}
/* Display the addr if a response was received */
if (!rc) {
console_printf("%02x ", addr);
dev_count++;
} else {
console_printf("-- ");
}
}
console_printf("\nFound %u devices on I2C bus 0\n", dev_count);

return 0;
   }

The HW (https://www.adafruit.com/product/1120 or 
https://www.adafruit.com/products/1714) has been used on about 20 
different MCUs by me, and it's usually one of the first drivers I write 
so I'm confident the HW is good.


What is unusual is that the LSM303DLHC is the only sensor that doesn't 
show up. We have 5 other I2C sensors connected to the test system that 
show up fine.


I thought it might be a HW issue we haven't seen before since we have 
5V/3V level shifting, but checking the LSM303DLHC datasheet I'm quite 
confident the HW breakout is fine and we are using the recommended 
values (10K pullups on SCL/SDA, etc.). It fails at both 100kHz and 400kHz.


I think the issue is ultimately here in the probe function with the STOP 
bit that is being sent (which I think is the right default). Changing it 
to not send the STOP condition does bring more devices up on the I2C bus:


   int
   hal_i2c_master_probe(uint8_t i2c_num, uint8_t address, uint32_t timo)
   {
struct hal_i2c_master_data rx;
uint8_t buf;

rx.address = address;
rx.buffer = 
rx.len = 1;

   *return hal_i2c_master_read(i2c_num, , timo, 0);*
   }

With the following sensors attached:

LSM303DLHC Accel: 0x19
LSM303DLHC Mag:   0x1E
TCS3472x: 0x29
FXAS21002C:   0x21
FXOS8700CQ:   0x1F
BMP280:   0x77
Si7201:   0x40

I get this with the default behaviour of the STOP bit being enabled ... 
everything show up except the LS303DLHC accel/mag:


   i2cscan
   12275:Scanning I2C bus 0
 0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
   00:  -- -- -- -- -- -- -- -- -- -- -- -- --
   10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 1f
   20: -- 21 -- -- -- -- -- -- -- 29 -- -- -- -- -- --
   30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   40: 40 -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   70: -- -- -- -- -- -- -- 77
   Found 5 devices on I2C bus 0


When I disable the STOP (lastop=0) bit I get more address, but still not 
the right ones and the next scan also adds 0x03:


   i2cscan
   7082:Scanning I2C bus 0
 0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
   00:  -- -- -- -- -- -- -- -- -- -- -- -- --
   10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 1f
   20: 20 21 22 -- -- -- -- -- -- 29 2a -- -- -- -- --
   30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   40: 40 41 -- -- -- -- -- -- -- -- -- -- -- -- -- --
   50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
   70: -- -- -- -- -- -- -- 77
   Found 9 devices on I2C bus 0


Probing the I2C bus is problematic anyway since there isn't a standard 
way to do it, although I think it is an important sanity check, and it's 
a pretty standard chunk of code on most systems I've used. I'm almost 
sure probing this way works on every other MCU I've tested, and I use 
this functionality all the time when writing drivers as an initial test 
to make sure the pins are connected properly. What's odd is that this 
seems to be unique to the nRF52, which makes me wonder if there is 
something specific about the I2C pad design on the new parts, and the 
LSM303DLHC happens to have a specific design that causes problems?


I think the right choice is sending the STOP bit, though, and the 

Re: WIP: sensor API

[Kevin Townsend]

On 18/12/16 23:50, Sterling Hughes wrote:

I will correct this, thanks.  I frequently forget that shell commands 
are not const.  I was too lazy to make them a linker section and const 
them, which I really should have done.
I've just started playing with this seriously tonight, and some of the 
values can be tweaked here and there, but overall the first pass at the 
sensor API is very nice.  Great job for a first pass, certainly.


The cfg system works very well since you can define a struct per driver, 
which solves a lot of issues and keeps things simple. You aren't locked 
into a situation where you have to work with the lowest common 
denominator of config settings and options, and can make drivers as 
basic or feature rich as you like. I think that's important.


One important thing that I believe should be added as mandatory 
meta-data for every driver, though, is 'driver version'. On long running 
systems that will see many firmware updates and device iterations or 
models, you probably want to know which version of the sensor driver you 
have, and maybe report this with sensor data when logging it so that you 
can compensate for any bugs that come up in the field. It can also be 
important with things like OIC where you may need to know not just which 
sensors are present, but which driver version to know is certain 
features are available or not. Just thinking out loud here, though. :)

Re: WIP: sensor API

[Kevin Townsend]

- 1. Read multiple values (default behaviour?)
- 2. Raise a missing sample alert


What is the failure condition when a sample is missed look like? My 
assumption was that this would be, for example, a FIFO overrun on a 
connected sensor that you are polling for data.


The main failure I had in mind was missing a polling interval because of 
resource allocation issues. Part of that is of course just planning, but 
there are DSP algorithms where missing an interval might have a 
significant influence on the output. But to be honest, I'm being overly 
paranoid here and if you have extremely strict timing requirements, the 
burden is on you to make sure you meet them by setting appropriate 
timing and priority values. And we do know when the last sample was read 
(off the top of my head), so we can probably detect an error already.


I guess another failure condition is that you are late in processing 
an interrupt (locally on the processor), or your task gets locked out 
for too long and you miss a poll.  However, I’m not sure if you want 
to be putting strict requirements on this task — I think we’d just 
want to log a delay / statistic every time we get locked out for more 
than ’n’ ms.
This was my main concern, yes. But I think as you implied if you have 
strict requirements, that should be part of the design and decisions 
from the start, and I don't know if we should bend over backwards in the 
sensor API to fit strict timing in.


K.

Re: WIP: sensor API

[Kevin Townsend]

There are probably instances where 1kHz isn't fast enough for certain 
sensor types:


https://github.com/apache/incubator-mynewt-core/compare/develop...sensors_branch#diff-ec052d973c26072d9ac2e198f16e764aR226

/**
 * Poll rate in MS for this sensor.
 */
uint32_t s_poll_rate;

Having said that ... if there are callbacks providing data events, we 
may not want that to fire at anything less than 1ms as well, so there is 
a tradeoff. At that point, perhaps collecting a number of data samples 
inside the driver may be valid, and then returning that collection of 
samples in the event callback, although this goes against the design 
decision not to buffer data internally?


Would a microsecond poll rate be too much to deal with on most systems 
in your opinion(s)? That still gives you a valid polling range of 
1us..4294s in a 32-bit value. That's only slightly more than an hour, 
though, which is a shame since I can imagine situations where you might 
want something every 6 hours as well. There may also be timer issues 
going below 1ms resolution as well.

Re: WIP: sensor API

[Kevin Townsend]

Hi Sterling,

One other important question is what happens when we don't respect the 
read request intervals between data samples?


In the accel sim (really nice to see sim examples, BTW ... that's 
extremely useful to test sensor algorithms), you just check the number 
of missing samples from the last interval and generate that many new 
samples to fill the gaps: 
https://github.com/apache/incubator-mynewt-core/compare/develop...sensors_branch#diff-1c17a623363565318dfefdb8891c0376R148


A lot of DSP algorithms are timing sensitive, though, such as sensor 
fusion with accel/mag/gyro inputs, where you want to samples as close as 
possible to a fixed rate like 50Hz or 100Hz, and the three components 
read together as closely as possible (though timestamp at least lest you 
know the difference between components).


Raising some sort of alert when a sample is missed might be a better 
choice, or having the option to indicate which of the two behaviours 
should be used by the timer when a sample interval is missed:


- 1. Read multiple values (default behaviour?)
- 2. Raise a missing sample alert
- 3. ???

Kevin

Re: I2C Pin Setup (nRF52)

[Kevin Townsend]

Same here


On 06/12/16 22:53, will sanfilippo wrote:

My votes, if we are taking a vote:

+1 on moving everything into hal_bsp.c
+1 on not using SDK definitions for the pins (those from nrf_drv_config.h).


On Dec 6, 2016, at 1:51 PM, Sterling Hughes <sterl...@apache.org> wrote:

Yeah, I think that was a oversight, because we didn’t have a standard early on.

I think everything should be in the C file & we should remove those definitions.

Sterling

On 6 Dec 2016, at 15:49, Kevin Townsend wrote:


There is a slight inconsistency here with UART which has the pins defined in 
syscfg.yml, whereas other peripherals are defined directly in hal_bsp.c:

UART_0:
description: 'TBD'
value:  1
UART_0_PIN_TX:
description: 'TBD'
value:  6
UART_0_PIN_RX:
description: 'TBD'
value:  8
UART_0_PIN_RTS:
description: 'TBD'
value:  0
UART_0_PIN_CTS:
description: 'TBD'
value: 0

There may well be reason(s) for this, but it might be easier to maintain to 
have everything in the .c file unless I'm missing something (entirely possible).


On 06/12/16 22:35, will sanfilippo wrote:

I think the issue here is when folks are using the nordic SDK as well as our 
HAL. The idea was that pin configurations would be in hal_bsp.c; nowhere else. 
When we moved out the nordic SDK, pin definitions from nrf_drv_config.h were 
moved into pkg.yml. This made for some conflicts.

I have an idea but not sure it is a good one. I will use the twi interface as 
an example; I think the rest of the nordic SDK is similar. Sorry to be 
“pedantic” here; but you know me :-)

Users can setup their own config structures and pass that config into the 
nordic SDK initialization. For example:

Here is the initialization function for twi. Note the *p_config structure:

ret_code_t nrf_drv_twi_init(nrf_drv_twi_t const *p_instance,
nrf_drv_twi_config_t const 
*p_config,
nrf_drv_twi_evt_handler_t
event_handler,
void *p_context)


Here is the structure:
/**
  * @brief Structure for the TWI master driver instance configuration.
  */
typedef struct
{
 uint32_tscl;   ///< SCL pin number.
 uint32_tsda;   ///< SDA pin number.
 nrf_twi_frequency_t frequency; ///< TWI frequency.
 uint8_t interrupt_priority;  ///< Interrupt priority.

} nrf_drv_twi_config_t;

In hal_bsp.c, someone would create this structure with pin defintions just like 
we do for our HAL and thus all pin defintions would be in hal_bsp.c. They then 
pass this structure in the init call. No more pin definitions in pkg.yml.

Another option which has been mentioned by others is to have a header file that 
contains all the pin definitions and this can be in the bsp. These definitions 
can be the same as the ones in nrf_drv_config.h or they can be user defined. My 
preference is to have them user-defined.

Hopefully this makes sense...

Will



On Dec 6, 2016, at 12:33 PM, Kevin Townsend <ke...@adafruit.com> wrote:

Hi,

I was trying to test out an I2C chip with a previously working driver to verify 
the HW, but it isn't clear anymore where to set pins (post Nordic SDK removal). 
I couldn't get I2C to work after tweaking the pkg.yml file which has a lot of 
pin defines, for example:

- '-DTWI0_CONFIG_SCL=26'
- '-DTWI0_CONFIG_SDA=25'

After triple checking plus resolving some conflicts where master and slave of 
the same bus types were set (in pkg.yml), etc., I looked at the hal_bsp.c file 
and see they are also hard coded there:

#if MYNEWT_VAL(I2C_0)
static const struct nrf52_hal_i2c_cfg hal_i2c_cfg = {
.scl_pin = 27,
.sda_pin = 26,
.i2c_frequency = 100/* 100 kHz */
};
#endif

It might be easier to make the hal_bsp.c here use the values from pkg.yml, or 
any solution really not to have two defines?

* 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/src/hal_bsp.c#L79
* 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/pkg.yml#L72

There are some conflicts as well like I'm guessing this isn't possible together?

* 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/pkg.yml#L74
* 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/pkg.yml#L76

Not trying to be a PITA ... but it was a bit confusing changing pins but not 
seeing the results. :)

I can send in a pull request as well if that's easier, but maybe other people 
here have a better idea of where you want all the pins defines to be between 
the .c file, syscfg.yml and pkg.yml?

I just couldn't get I2C to work, for example with this scanner I put together 
which worked fine previously:

static int
shell_i2cscan_cmd(int argc, char **argv)
{
uint8_t addr;
int32_t timeout = OS_TICKS_PER_SEC / 10;
uint8_t dev_co

Re: I2C Pin Setup (nRF52)

[Kevin Townsend]

There is a slight inconsistency here with UART which has the pins 
defined in syscfg.yml, whereas other peripherals are defined directly in 
hal_bsp.c:


UART_0:
description: 'TBD'
value:  1
UART_0_PIN_TX:
description: 'TBD'
value:  6
UART_0_PIN_RX:
description: 'TBD'
value:  8
UART_0_PIN_RTS:
description: 'TBD'
value:  0
UART_0_PIN_CTS:
description: 'TBD'
value: 0

There may well be reason(s) for this, but it might be easier to maintain 
to have everything in the .c file unless I'm missing something (entirely 
possible).



On 06/12/16 22:35, will sanfilippo wrote:

I think the issue here is when folks are using the nordic SDK as well as our 
HAL. The idea was that pin configurations would be in hal_bsp.c; nowhere else. 
When we moved out the nordic SDK, pin definitions from nrf_drv_config.h were 
moved into pkg.yml. This made for some conflicts.

I have an idea but not sure it is a good one. I will use the twi interface as 
an example; I think the rest of the nordic SDK is similar. Sorry to be 
“pedantic” here; but you know me :-)

Users can setup their own config structures and pass that config into the 
nordic SDK initialization. For example:

Here is the initialization function for twi. Note the *p_config structure:

ret_code_t nrf_drv_twi_init(nrf_drv_twi_t const *p_instance,
nrf_drv_twi_config_t const 
*p_config,
nrf_drv_twi_evt_handler_t
event_handler,
void *p_context)


Here is the structure:
/**
  * @brief Structure for the TWI master driver instance configuration.
  */
typedef struct
{
 uint32_tscl;   ///< SCL pin number.
 uint32_tsda;   ///< SDA pin number.
 nrf_twi_frequency_t frequency; ///< TWI frequency.
 uint8_t interrupt_priority;  ///< Interrupt priority.

} nrf_drv_twi_config_t;

In hal_bsp.c, someone would create this structure with pin defintions just like 
we do for our HAL and thus all pin defintions would be in hal_bsp.c. They then 
pass this structure in the init call. No more pin definitions in pkg.yml.

Another option which has been mentioned by others is to have a header file that 
contains all the pin definitions and this can be in the bsp. These definitions 
can be the same as the ones in nrf_drv_config.h or they can be user defined. My 
preference is to have them user-defined.

Hopefully this makes sense...

Will



On Dec 6, 2016, at 12:33 PM, Kevin Townsend <ke...@adafruit.com> wrote:

Hi,

I was trying to test out an I2C chip with a previously working driver to verify 
the HW, but it isn't clear anymore where to set pins (post Nordic SDK removal). 
I couldn't get I2C to work after tweaking the pkg.yml file which has a lot of 
pin defines, for example:

- '-DTWI0_CONFIG_SCL=26'
- '-DTWI0_CONFIG_SDA=25'

After triple checking plus resolving some conflicts where master and slave of 
the same bus types were set (in pkg.yml), etc., I looked at the hal_bsp.c file 
and see they are also hard coded there:

#if MYNEWT_VAL(I2C_0)
static const struct nrf52_hal_i2c_cfg hal_i2c_cfg = {
.scl_pin = 27,
.sda_pin = 26,
.i2c_frequency = 100/* 100 kHz */
};
#endif

It might be easier to make the hal_bsp.c here use the values from pkg.yml, or 
any solution really not to have two defines?

* 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/src/hal_bsp.c#L79
* 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/pkg.yml#L72

There are some conflicts as well like I'm guessing this isn't possible together?

* 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/pkg.yml#L74
* 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/pkg.yml#L76

Not trying to be a PITA ... but it was a bit confusing changing pins but not 
seeing the results. :)

I can send in a pull request as well if that's easier, but maybe other people 
here have a better idea of where you want all the pins defines to be between 
the .c file, syscfg.yml and pkg.yml?

I just couldn't get I2C to work, for example with this scanner I put together 
which worked fine previously:

static int
shell_i2cscan_cmd(int argc, char **argv)
{
uint8_t addr;
int32_t timeout = OS_TICKS_PER_SEC / 10;
uint8_t dev_count = 0;

console_printf("Scanning I2C bus 0\n"
   " 0  1  2  3  4  5  6  7  8  9  a  b  c  d e  f\n"
   "00:  ");

/* Scan all valid I2C addresses (0x03..0x77) */
for (addr = 0x03; addr < 0x78; addr++) {
int rc = hal_i2c_master_probe(0, addr, timeout);
/* Print addr header every 16 bytes */
if (!(addr % 16)) {
  console_printf("\n%02x: ", addr);
}
/* D

I2C Pin Setup (nRF52)

[Kevin Townsend]

Hi,

I was trying to test out an I2C chip with a previously working driver to 
verify the HW, but it isn't clear anymore where to set pins (post Nordic 
SDK removal). I couldn't get I2C to work after tweaking the pkg.yml file 
which has a lot of pin defines, for example:


- '-DTWI0_CONFIG_SCL=26'
- '-DTWI0_CONFIG_SDA=25'

After triple checking plus resolving some conflicts where master and 
slave of the same bus types were set (in pkg.yml), etc., I looked at the 
hal_bsp.c file and see they are also hard coded there:


#if MYNEWT_VAL(I2C_0)
static const struct nrf52_hal_i2c_cfg hal_i2c_cfg = {
.scl_pin = 27,
.sda_pin = 26,
.i2c_frequency = 100/* 100 kHz */
};
#endif

It might be easier to make the hal_bsp.c here use the values from 
pkg.yml, or any solution really not to have two defines?


 * 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/src/hal_bsp.c#L79
 * 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/pkg.yml#L72

There are some conflicts as well like I'm guessing this isn't possible 
together?


 * 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/pkg.yml#L74
 * 
https://github.com/apache/incubator-mynewt-core/blob/master/hw/bsp/nrf52dk/pkg.yml#L76

Not trying to be a PITA ... but it was a bit confusing changing pins but 
not seeing the results. :)


I can send in a pull request as well if that's easier, but maybe other 
people here have a better idea of where you want all the pins defines to 
be between the .c file, syscfg.yml and pkg.yml?


I just couldn't get I2C to work, for example with this scanner I put 
together which worked fine previously:


static int
shell_i2cscan_cmd(int argc, char **argv)
{
uint8_t addr;
int32_t timeout = OS_TICKS_PER_SEC / 10;
uint8_t dev_count = 0;

console_printf("Scanning I2C bus 0\n"
   " 0  1  2  3  4  5  6  7  8  9  a  b  c  d e  f\n"
   "00:  ");

/* Scan all valid I2C addresses (0x03..0x77) */
for (addr = 0x03; addr < 0x78; addr++) {
int rc = hal_i2c_master_probe(0, addr, timeout);
/* Print addr header every 16 bytes */
if (!(addr % 16)) {
  console_printf("\n%02x: ", addr);
}
/* Display the addr if a response was received */
if (!rc) {
console_printf("%02x ", addr);
dev_count++;
} else {
console_printf("-- ");
}
}
console_printf("\nFound %u devices on I2C bus 0\n", dev_count);

return 0;
}

Kevin

Re: MBEDTLS Error after 'newt install -v'

[Kevin Townsend]

Updating the project.yml file to point to 0.0.1 resolved this, and a 
clean setup now pulls master instead of develop.


I agree minimum version checks would be extremely useful, though, since 
the newt/newtmgr tools and apache-mynewt-core are quite tightly coupled.



On 05/12/16 18:29, Kevin Townsend wrote:

Hi Chris,

Thanks for the reply. I usually pull the tools and mynewt core at the 
same time. We're running the latest version of both repos from master, 
or at least that's what I thought ... it seems when I did a 'newt 
install -v' we ended up in 'develop' in repos/apache-mynewt-core


This is most likely an artifact with our local mynewt project.yml or 
project.state file or something similar in our internal repo:


repository.apache-mynewt-core:
type: github
vers: 0-dev
user: apache
repo: incubator-mynewt-core

Apparently this should be 0.0.1 now? 
https://github.com/apache/incubator-mynewt-core/blob/master/repository.yml


K.


On 05/12/16 18:24, Christopher Collins wrote:

Hi Kevin,

This issue tends to happen if you are using a version of newt that is
incompatible with your core repo.  You mentioned you are using the
master branch of core; is it possible that you are using develop of the
newt repo?

Specifically, I think this issue is caused by this commit from last
week (newt repo): 0241d2f38e8236f03d6f20d5c7198d4a0a4cb169

I mentioned in an email last week that newt and core would need to be
upgraded together after this change.  Emails like that aren't
particularly useful, because they are only interesting to people who are
already experiencing issues!

One feature that newt needs is the ability to require minimum versions
of the core repo.  Then, compatibility-breaking changes would still be a
nuissance, but at least it would be obvious what the problem is.

Chris

On Mon, Dec 05, 2016 at 06:11:24PM +0100, Kevin Townsend wrote:
I was updating some local dev systems, and with a clean install and 
then

running 'newt install -v' we now get the following error when we try to
build the bootloader:

/LOCAL_FOLDER/repos/apache-mynewt-core/crypto/mbedtls/include/mbedtls/config.h:2522:10: 


error: #include expects "FILENAME" or 
   #include MBEDTLS_USER_CONFIG_FILE
^

This is new, but unexpected since nothing has changed in the last week
or so in the master branch of the repo, and the last change to 'boot'
was 26 days ago so I'm not missing something in our local slightly
modified bootloader code.

Has something recently changed with the mbedtls library that I'm not
seeing quickly looking at the git history in master?

Just commenting the offending line out causes problems elsewhere so
there seems to be a missing config file:

https://github.com/apache/incubator-mynewt-core/blob/master/crypto/mbedtls/include/mbedtls/config.h#L2522 



 #if defined(YOTTA_CFG_MBEDTLS_USER_CONFIG_FILE)
 #include YOTTA_CFG_MBEDTLS_USER_CONFIG_FILE
 #elif defined(MBEDTLS_USER_CONFIG_FILE)
 //#include MBEDTLS_USER_CONFIG_FILE
 #endif

That just kicks the problems further down the field:

 ctr_drbg.c:425:5: error: implicit declaration of function 'fopen'
 [-Werror=implicit-function-declaration]
   if( ( f = fopen( path, "wb" ) ) == NULL )
   ^
 ctr_drbg.c:425:13: error: assignment makes pointer from integer
 without a cast [-Werror]
   if( ( f = fopen( path, "wb" ) ) == NULL )
   ^
 ctr_drbg.c:440:5: error: implicit declaration of function 'fclose'
 [-Werror=implicit-function-declaration]
   fclose( f );
   ^
 ctr_drbg.c: In function 'mbedtls_ctr_drbg_update_seed_file':
 ctr_drbg.c:450:13: error: assignment makes pointer from integer
 without a cast [-Werror]
   if( ( f = fopen( path, "rb" ) ) == NULL )
   ^
 ctr_drbg.c:453:5: error: implicit declaration of function 'fseek'
 [-Werror=implicit-function-declaration]
   fseek( f, 0, SEEK_END );
   ^
 ctr_drbg.c:453:18: error: 'SEEK_END' undeclared (first use in this
 function)
   fseek( f, 0, SEEK_END );
^
 ctr_drbg.c:453:18: note: each undeclared identifier is reported 
only

 once for each function it appears in
 ctr_drbg.c:454:5: error: implicit declaration of function 'ftell'
 [-Werror=implicit-function-declaration]
   n = (size_t) ftell( f );
   ^
 ctr_drbg.c:455:18: error: 'SEEK_SET' undeclared (first use in this
 function)
   fseek( f, 0, SEEK_SET );
^
 cc1: all warnings being treated as errors

Any idea what might be causing the issue here? I'm looking at the
mbedtls source and don't see any recent pushes there, etc.

Re: MBEDTLS Error after 'newt install -v'

[Kevin Townsend]

Hi Chris,

Thanks for the reply. I usually pull the tools and mynewt core at the 
same time. We're running the latest version of both repos from master, 
or at least that's what I thought ... it seems when I did a 'newt 
install -v' we ended up in 'develop' in repos/apache-mynewt-core


This is most likely an artifact with our local mynewt project.yml or 
project.state file or something similar in our internal repo:


repository.apache-mynewt-core:
type: github
vers: 0-dev
user: apache
repo: incubator-mynewt-core

Apparently this should be 0.0.1 now? 
https://github.com/apache/incubator-mynewt-core/blob/master/repository.yml


K.


On 05/12/16 18:24, Christopher Collins wrote:

Hi Kevin,

This issue tends to happen if you are using a version of newt that is
incompatible with your core repo.  You mentioned you are using the
master branch of core; is it possible that you are using develop of the
newt repo?

Specifically, I think this issue is caused by this commit from last
week (newt repo): 0241d2f38e8236f03d6f20d5c7198d4a0a4cb169

I mentioned in an email last week that newt and core would need to be
upgraded together after this change.  Emails like that aren't
particularly useful, because they are only interesting to people who are
already experiencing issues!

One feature that newt needs is the ability to require minimum versions
of the core repo.  Then, compatibility-breaking changes would still be a
nuissance, but at least it would be obvious what the problem is.

Chris

On Mon, Dec 05, 2016 at 06:11:24PM +0100, Kevin Townsend wrote:

I was updating some local dev systems, and with a clean install and then
running 'newt install -v' we now get the following error when we try to
build the bootloader:

/LOCAL_FOLDER/repos/apache-mynewt-core/crypto/mbedtls/include/mbedtls/config.h:2522:10:
error: #include expects "FILENAME" or 
   #include MBEDTLS_USER_CONFIG_FILE
^

This is new, but unexpected since nothing has changed in the last week
or so in the master branch of the repo, and the last change to 'boot'
was 26 days ago so I'm not missing something in our local slightly
modified bootloader code.

Has something recently changed with the mbedtls library that I'm not
seeing quickly looking at the git history in master?

Just commenting the offending line out causes problems elsewhere so
there seems to be a missing config file:

https://github.com/apache/incubator-mynewt-core/blob/master/crypto/mbedtls/include/mbedtls/config.h#L2522

 #if defined(YOTTA_CFG_MBEDTLS_USER_CONFIG_FILE)
 #include YOTTA_CFG_MBEDTLS_USER_CONFIG_FILE
 #elif defined(MBEDTLS_USER_CONFIG_FILE)
 //#include MBEDTLS_USER_CONFIG_FILE
 #endif

That just kicks the problems further down the field:

 ctr_drbg.c:425:5: error: implicit declaration of function 'fopen'
 [-Werror=implicit-function-declaration]
   if( ( f = fopen( path, "wb" ) ) == NULL )
   ^
 ctr_drbg.c:425:13: error: assignment makes pointer from integer
 without a cast [-Werror]
   if( ( f = fopen( path, "wb" ) ) == NULL )
   ^
 ctr_drbg.c:440:5: error: implicit declaration of function 'fclose'
 [-Werror=implicit-function-declaration]
   fclose( f );
   ^
 ctr_drbg.c: In function 'mbedtls_ctr_drbg_update_seed_file':
 ctr_drbg.c:450:13: error: assignment makes pointer from integer
 without a cast [-Werror]
   if( ( f = fopen( path, "rb" ) ) == NULL )
   ^
 ctr_drbg.c:453:5: error: implicit declaration of function 'fseek'
 [-Werror=implicit-function-declaration]
   fseek( f, 0, SEEK_END );
   ^
 ctr_drbg.c:453:18: error: 'SEEK_END' undeclared (first use in this
 function)
   fseek( f, 0, SEEK_END );
^
 ctr_drbg.c:453:18: note: each undeclared identifier is reported only
 once for each function it appears in
 ctr_drbg.c:454:5: error: implicit declaration of function 'ftell'
 [-Werror=implicit-function-declaration]
   n = (size_t) ftell( f );
   ^
 ctr_drbg.c:455:18: error: 'SEEK_SET' undeclared (first use in this
 function)
   fseek( f, 0, SEEK_SET );
^
 cc1: all warnings being treated as errors

Any idea what might be causing the issue here? I'm looking at the
mbedtls source and don't see any recent pushes there, etc.

Re: MBEDTLS Error after 'newt install -v'

[Kevin Townsend]

Just as a follow up, for some reason a clean install set me to 'develop' 
not 'master', though it isn't clear to me why yet ... but this resolves 
the issue with the missing config file:


 * cd repos
 * cd apache-mynewt-core
 * git branch # To show that I'm on develop not master which is what I
   expected (could be a local issue on my end!)
 * git checkout master
 * cd ../..
 * newt build bootloader


On 05/12/16 18:11, Kevin Townsend wrote:


I was updating some local dev systems, and with a clean install and 
then running 'newt install -v' we now get the following error when we 
try to build the bootloader:


/LOCAL_FOLDER/repos/apache-mynewt-core/crypto/mbedtls/include/mbedtls/config.h:2522:10: 
error: #include expects "FILENAME" or 

 #include MBEDTLS_USER_CONFIG_FILE
  ^

This is new, but unexpected since nothing has changed in the last week 
or so in the master branch of the repo, and the last change to 'boot' 
was 26 days ago so I'm not missing something in our local slightly 
modified bootloader code.


Has something recently changed with the mbedtls library that I'm not 
seeing quickly looking at the git history in master?


Just commenting the offending line out causes problems elsewhere so 
there seems to be a missing config file:


https://github.com/apache/incubator-mynewt-core/blob/master/crypto/mbedtls/include/mbedtls/config.h#L2522

#if defined(YOTTA_CFG_MBEDTLS_USER_CONFIG_FILE)
#include YOTTA_CFG_MBEDTLS_USER_CONFIG_FILE
#elif defined(MBEDTLS_USER_CONFIG_FILE)
//#include MBEDTLS_USER_CONFIG_FILE
#endif

That just kicks the problems further down the field:

ctr_drbg.c:425:5: error: implicit declaration of function 'fopen'
[-Werror=implicit-function-declaration]
 if( ( f = fopen( path, "wb" ) ) == NULL )
 ^
ctr_drbg.c:425:13: error: assignment makes pointer from integer
without a cast [-Werror]
 if( ( f = fopen( path, "wb" ) ) == NULL )
 ^
ctr_drbg.c:440:5: error: implicit declaration of function 'fclose'
[-Werror=implicit-function-declaration]
 fclose( f );
 ^
ctr_drbg.c: In function 'mbedtls_ctr_drbg_update_seed_file':
ctr_drbg.c:450:13: error: assignment makes pointer from integer
without a cast [-Werror]
 if( ( f = fopen( path, "rb" ) ) == NULL )
 ^
ctr_drbg.c:453:5: error: implicit declaration of function 'fseek'
[-Werror=implicit-function-declaration]
 fseek( f, 0, SEEK_END );
 ^
ctr_drbg.c:453:18: error: 'SEEK_END' undeclared (first use in this
function)
 fseek( f, 0, SEEK_END );
  ^
ctr_drbg.c:453:18: note: each undeclared identifier is reported
only once for each function it appears in
ctr_drbg.c:454:5: error: implicit declaration of function 'ftell'
[-Werror=implicit-function-declaration]
 n = (size_t) ftell( f );
 ^
ctr_drbg.c:455:18: error: 'SEEK_SET' undeclared (first use in this
function)
 fseek( f, 0, SEEK_SET );
  ^
cc1: all warnings being treated as errors

Any idea what might be causing the issue here? I'm looking at the 
mbedtls source and don't see any recent pushes there, etc.

[newtmgr] Delete/overwrite image in slot 1

[Kevin Townsend]

Is there a means with newtmgr today to delete an image in bank 1, for 
example if you uploaded an image there incorrectly?


Assuming the following situation:

Images:
 slot=0
  version: 0.3.0
  bootable: true
  flags: active
  hash: 7494da56ead5fe83a0e26ad72fdf1bc8a4d1d6b01b1712e4e1bf4437218f4d8b
 slot=1
  version: 0.2.0
  bootable: true
  flags: confirmed
  hash: bd2fb0d9b2c771b284b19d7c143754a8e26cfae837f1e50bb2467ee8dd57e9da

If you try to upload an image to bank 1 again you will get this error:

newtmgr -c serial image upload 
/Users/antonio/Development/Adafruit/NewtManager/imgfiles/bleuart_0_3_0.img

Error: Target error: 2

newtmgr -c serial image upload 
/Users/antonio/Development/Adafruit/NewtManager/imgfiles/throughput_0_2_0.img 


Error: Target error: 2

Would it make sense to have an option to clear the second bank as an 
option via a specific newtmgr command, or perhaps there are some 
specific reasons to prevent the second image from being changed or 
overwritten that I'm not aware of?

Re: OIC support

[Kevin Townsend]

me and Paul brought in constrained iotivity stack few months back
to Mynewt, and I probably should send an update what’s going on.

I’ve been working on getting it more fully integrated. My first
goal is to get the code size/RAM usage down.

Just as a FYI for anyone else interested in understanding 
OIC/Iotivity/etc. a bit more, I found this presentation useful to 
highlight the key concepts: 
https://openconnectivity.org/wp-content/uploads/2016/01/OIC_Specification_Overview_201501131.pdf

Re: OIC support

[Kevin Townsend]

That is a good point. There are data models for different
type of sensors/controls over there. I’m see these are at
http://oneiota.org . Acceleration, humidity, temperature 
etc.

This is very useful, and it should be easy to add another layer on top 
of the base driver to return data in an OIC appropriate format, but this 
at least solves the confusion around SI unit types and scale, etc.

- Are there specific config mechanisms that should be kept in mind?

Current framework provides a way to register resources; you can
register put/get routines for sensor/control. So it would be possible
to have very specific code per sensor.

However, if our sensor framework allowed us to enumerate connected
sensors, with common read interface serving data in known format, we
could export connected sensors to OIC automatically. That might be pretty
exciting.
I think that's what everyone had in mind, yes, and similar to what I've 
done with other systems where I implemented a generic 'read' and 'get 
sensor details' function and a single sample data type across all 
sensors (following what Android does beneath the hood).


With a bit of thought and work, it should be possible to scan the I2C 
bus at startup (for example) and initialise any sensors found on the 
system if desirable, although it does mean maintaining a large list of 
drivers in the firmware so space will become an issue over time.


Thanks for the link above though  it's very useful to start poking 
at a proof of concept, and keep in mind for any first draft APIs.

Re: Mynewt iOS App

[Kevin Townsend]

On 01/12/16 21:35, David G. Simmons wrote:



Rather than go through every Bluetooth device within range, connect to 
it, and see if it's offering the right service, I simply check the 
name of the device and, if it's not a mynewt device, move on. This is 
faster than connecting to each device.


Normally you would include a specific service UUID in the advertising 
packet to solve this problem, and when you collect the advertising data 
you can determine if the service you care about is present, which might 
be less arbitrary than relying on a device name? 128-bit UUIDs do cause 
a problem, but you can still fit one in the main adv. packet, or you can 
optionally use the scan response for a second payload.


Just a suggestion about a fairly standard way to solve this problem 
without having to connect to every device in range. :)


Kevin

Re: OIC support

[Kevin Townsend]

Hi,

me and Paul brought in constrained iotivity stack few months back
to Mynewt, and I probably should send an update what’s going on.

I’ve been working on getting it more fully integrated. My first
goal is to get the code size/RAM usage down.

To cut down code size I’m converting it to use MyNewt’s memory pools,
timers, and event queues. By default we also left quite a bit logging
turned on, so I’ll make that optional (inclusion controlled by syscfg
settings).
To cut down RAM use I’m going to convert it to use os_mbuf for
data packets. This will probably make code size slightly larger,
but allows me to share the memory pool with bluetooth stack.

My end goal is to have OIC running over bluetooth in a way which
allows us to connect to apps written using iotivity’s framework
https://github.com/iotivity/iotivity .
I'm quite interested in this from a sensor driver point of view. I have 
no experience with iotivity, but if it will be a core part of Mynewt 
moving forward, it would be good to get a sense of what kinds of 
requirements the stack would place on sensors and sensor configuration, 
since that's the area I'm most likely to get involved with on my end.


- Are there specific config mechanisms that should be kept in mind?
- Are there any definitions about power mode and device/driver 'state'?
- What are the practical payload limitations given BLE as a transport?

It's too early to probably answer a lot of those things definitively, 
but it would be good to get the ball rolling as we start throwing some 
ideas around publicly about a sensor API as well.


Kevin

Re: HEX DEBUG output in newtmgr

[Kevin Townsend]

I think even just converting these 'data' and 'into buffer' DEC payloads 
to HEX is good enough:


[2 0 0 10 0 0 0 0 161 97 100 102 97 98 99 49 50 51]

The other values in base10 are fine in my opinion, and probably even 
more useful that way:


Op:2 Flags:0 Len:10 Group:0 Seq:0 Id:0

And then the full HEX + ASCII dump at the bottom is perfect



On 30/11/16 17:55, Sterling Hughes wrote:

captured here: https://issues.apache.org/jira/browse/MYNEWT-497

i don’t see a reason to print both hex and decimal, although if 
someone cares, retaining it as an option seems fine.  i agree that 
default should be hex.


On 30 Nov 2016, at 8:52, Kevin Townsend wrote:


Hi Christopher,

The latest version has improved debug output with the combined HEX 
and DEC which I don't remember in earlier versions, so thanks for 
that. We've been using the -t trace option but it's good to point out 
to anyone who stumbles across this thread anyway.


I don't find the DEC very useful in the [DEBUG] output lines within 
'Data:' which is what I was referring to in the original email, but 
the TX packet dump is a nice approach where you list both HEX and 
ASCII. My brain is just hard-wired to HEX and I find myself 
converting the verbose DEC values on top (which are very useful for 
the labels) into the HEX payload below to match things up. 100% HEX 
might be more natural for a lot of people.


Between -lDEBUG and -t we were able to get everything we needed and 
there are higher priority things to cleanup, and I'm happy to have 
what is there today, but I'd say just dropping DEC altogether 
probably isn't going to cause any problems ... but perhaps other 
users will disagree???



On 30/11/16 17:42, Christopher Collins wrote:

I accidently wrapped the newtmgr output with short lines.  Here is the
unadalterated output:

[ccollins@ccollins-mac:~/releases/1.0.0-b1/rc2]$ newtmgr -ldebug -c 
ttys010 echo abc123
2016/11/30 08:25:57 [DEBUG] Writing newtmgr request &{Op:2 Flags:0 
Len:10 Group:0 Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]}
2016/11/30 08:25:57 [DEBUG] Serializing request &{Op:2 Flags:0 
Len:10 Group:0 Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]} 
into buffer [2 0 0 10 0 0 0 0 161 97 100 102 97 98 99 49 50 51]

2016/11/30 08:25:57 [DEBUG] Tx packet dump:
  02 00 00 0a 00 00 00 00  a1 61 64 66 61 62 63 31 
|.adfabc1|

0010  32 33 |23|

2016/11/30 08:25:57 [DEBUG] Writing [6 9] to data channel
2016/11/30 08:25:57 [DEBUG] Writing [65 66 81 67 65 65 65 75 65 65 
65 65 65 75 70 104 90 71 90 104 89 109 77 120 77 106 79 53 57 65 61 
61] to data channel

2016/11/30 08:25:57 [DEBUG] Writing [10] to data channel
^C
[ccollins@ccollins-mac:~/releases/1.0.0-b1/rc2]$ newtmgr -t -ldebug 
-c ttys010 echo abc123
2016/11/30 08:26:13 [DEBUG] Writing newtmgr request &{Op:2 Flags:0 
Len:10 Group:0 Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]}
2016/11/30 08:26:13 [DEBUG] Serializing request &{Op:2 Flags:0 
Len:10 Group:0 Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]} 
into buffer [2 0 0 10 0 0 0 0 161 97 100 102 97 98 99 49 50 51]

2016/11/30 08:26:13 [DEBUG] Tx packet dump:
  02 00 00 0a 00 00 00 00  a1 61 64 66 61 62 63 31 
|.adfabc1|

0010  32 33 |23|

2016/11/30 08:26:13 [INFO] Outgoing:
  06 09 |..|

2016/11/30 08:26:13 [DEBUG] Writing [6 9] to data channel
2016/11/30 08:26:13 [INFO] Outgoing:
  41 42 51 43 41 41 41 4b  41 41 41 41 41 4b 46 68 
|ABQCAAAKAKFh|
0010  5a 47 5a 68 59 6d 4d 78  4d 6a 4f 35 39 41 3d 3d 
|ZGZhYmMxMjO59A==|


2016/11/30 08:26:13 [DEBUG] Writing [65 66 81 67 65 65 65 75 65 65 
65 65 65 75 70 104 90 71 90 104 89 109 77 120 77 106 79 53 57 65 61 
61] to data channel

2016/11/30 08:26:13 [INFO] Outgoing:
  0a |.|

2016/11/30 08:26:13 [DEBUG] Writing [10] to data channel
^C

Thanks,
Chris

On Wed, Nov 30, 2016 at 08:40:39AM -0800, Christopher Collins wrote:

Hi Kevin,

On Wed, Nov 30, 2016 at 01:37:38PM +0100, Kevin Townsend wrote:

We're currently working on a mobile version of newtmgr, and the DEBUG
output is very useful to sanity check commands and responses, but the
output in DEC is kind of a pain when most SW engineering on embedded
devices is done in HEX. Is there an option I'm missing to output HEX
instead of DEC, or is this something worth considering changing
internally in the tool to default to HEX which feels like a far more
natural choice from my PoV?

The newtmgr debug output is pretty much a mess at the moment.  That
said, it does do a hex dump of incoming and outgoing packets when you
specify the "-ldebug" flag.  If you are only seeing decimal output, 
then

is it possible you are using an old version of newtmgr?

The reason I say the debug output is a mess is that it prints both a
decimal and a hex dump.  For example:

 [ccollins@ccollins-mac:~/releases/1.0.0-b1/rc2]$ newtmgr 
-ldebug -c

 ttys010 echo abc123
 2016/11/30 08:25:57 [DEBUG] Writing newtmgr request &{Op:2 
F

Re: HEX DEBUG output in newtmgr

[Kevin Townsend]

Hi Christopher,

The latest version has improved debug output with the combined HEX and 
DEC which I don't remember in earlier versions, so thanks for that. 
We've been using the -t trace option but it's good to point out to 
anyone who stumbles across this thread anyway.


I don't find the DEC very useful in the [DEBUG] output lines within 
'Data:' which is what I was referring to in the original email, but the 
TX packet dump is a nice approach where you list both HEX and ASCII. My 
brain is just hard-wired to HEX and I find myself converting the verbose 
DEC values on top (which are very useful for the labels) into the HEX 
payload below to match things up. 100% HEX might be more natural for a 
lot of people.


Between -lDEBUG and -t we were able to get everything we needed and 
there are higher priority things to cleanup, and I'm happy to have what 
is there today, but I'd say just dropping DEC altogether probably isn't 
going to cause any problems ... but perhaps other users will disagree???



On 30/11/16 17:42, Christopher Collins wrote:

I accidently wrapped the newtmgr output with short lines.  Here is the
unadalterated output:

[ccollins@ccollins-mac:~/releases/1.0.0-b1/rc2]$ newtmgr -ldebug -c ttys010 
echo abc123
2016/11/30 08:25:57 [DEBUG] Writing newtmgr request &{Op:2 Flags:0 Len:10 
Group:0 Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]}
2016/11/30 08:25:57 [DEBUG] Serializing request &{Op:2 Flags:0 Len:10 Group:0 
Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]} into buffer [2 0 0 10 0 0 0 0 
161 97 100 102 97 98 99 49 50 51]
2016/11/30 08:25:57 [DEBUG] Tx packet dump:
  02 00 00 0a 00 00 00 00  a1 61 64 66 61 62 63 31  |.adfabc1|
0010  32 33 |23|

2016/11/30 08:25:57 [DEBUG] Writing [6 9] to data channel
2016/11/30 08:25:57 [DEBUG] Writing [65 66 81 67 65 65 65 75 65 65 65 65 65 75 
70 104 90 71 90 104 89 109 77 120 77 106 79 53 57 65 61 61] to data channel
2016/11/30 08:25:57 [DEBUG] Writing [10] to data channel
^C
[ccollins@ccollins-mac:~/releases/1.0.0-b1/rc2]$ newtmgr -t -ldebug -c ttys010 
echo abc123
2016/11/30 08:26:13 [DEBUG] Writing newtmgr request &{Op:2 Flags:0 Len:10 
Group:0 Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]}
2016/11/30 08:26:13 [DEBUG] Serializing request &{Op:2 Flags:0 Len:10 Group:0 
Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]} into buffer [2 0 0 10 0 0 0 0 
161 97 100 102 97 98 99 49 50 51]
2016/11/30 08:26:13 [DEBUG] Tx packet dump:
  02 00 00 0a 00 00 00 00  a1 61 64 66 61 62 63 31  |.adfabc1|
0010  32 33 |23|

2016/11/30 08:26:13 [INFO] Outgoing:
  06 09 |..|

2016/11/30 08:26:13 [DEBUG] Writing [6 9] to data channel
2016/11/30 08:26:13 [INFO] Outgoing:
  41 42 51 43 41 41 41 4b  41 41 41 41 41 4b 46 68  |ABQCAAAKAKFh|
0010  5a 47 5a 68 59 6d 4d 78  4d 6a 4f 35 39 41 3d 3d  |ZGZhYmMxMjO59A==|

2016/11/30 08:26:13 [DEBUG] Writing [65 66 81 67 65 65 65 75 65 65 65 65 65 75 
70 104 90 71 90 104 89 109 77 120 77 106 79 53 57 65 61 61] to data channel
2016/11/30 08:26:13 [INFO] Outgoing:
  0a|.|

2016/11/30 08:26:13 [DEBUG] Writing [10] to data channel
^C

Thanks,
Chris

On Wed, Nov 30, 2016 at 08:40:39AM -0800, Christopher Collins wrote:

Hi Kevin,

On Wed, Nov 30, 2016 at 01:37:38PM +0100, Kevin Townsend wrote:

We're currently working on a mobile version of newtmgr, and the DEBUG
output is very useful to sanity check commands and responses, but the
output in DEC is kind of a pain when most SW engineering on embedded
devices is done in HEX. Is there an option I'm missing to output HEX
instead of DEC, or is this something worth considering changing
internally in the tool to default to HEX which feels like a far more
natural choice from my PoV?

The newtmgr debug output is pretty much a mess at the moment.  That
said, it does do a hex dump of incoming and outgoing packets when you
specify the "-ldebug" flag.  If you are only seeing decimal output, then
is it possible you are using an old version of newtmgr?

The reason I say the debug output is a mess is that it prints both a
decimal and a hex dump.  For example:

 [ccollins@ccollins-mac:~/releases/1.0.0-b1/rc2]$ newtmgr -ldebug -c
 ttys010 echo abc123
 2016/11/30 08:25:57 [DEBUG] Writing newtmgr request &{Op:2 Flags:0
 Len:10 Group:0 Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]}
 2016/11/30 08:25:57 [DEBUG] Serializing request &{Op:2 Flags:0 Len:10
 Group:0 Seq:0 Id:0 Data:[161 97 100 102 97 98 99 49 50 51]} into buffer
 [2 0 0 10 0 0 0 0 161 97 100 102 97 98 99 49 50 51]
 2016/11/30 08:25:57 [DEBUG] Tx packet dump:
   02 00 00 0a 00 00 00 00  a1 61 64 66 61 62 63 31
 |.adfabc1|
 0010  32 33 |23

HEX DEBUG output in newtmgr

[Kevin Townsend]

We're currently working on a mobile version of newtmgr, and the DEBUG 
output is very useful to sanity check commands and responses, but the 
output in DEC is kind of a pain when most SW engineering on embedded 
devices is done in HEX. Is there an option I'm missing to output HEX 
instead of DEC, or is this something worth considering changing 
internally in the tool to default to HEX which feels like a far more 
natural choice from my PoV?

Re: [VOTE] Release Apache Mynewt 1.0.0-b1-incubating-rc1

[Kevin Townsend]

+1 from me, though I've mostly only been kicking the tires in on the 
HAL. But no glaring blocking issues there on my part so far, and things 
have been running reliable so far.



On 23/11/16 16:05, Jim Jagielski wrote:

+1 (binding)


On Nov 22, 2016, at 8:58 PM, Christopher Collins  wrote:

Hello all,

I am pleased to be calling this vote for the source release of Apache
Mynewt 1.0.0, beta 1.

Apache Mynewt is a community-driven, permissively licensed open source
initiative for constrained, embedded applications. Mynewt provides a
real-time operating system, flash file system, network stacks, and
support utilities for real-world embedded systems.

For full release notes, please visit the Apache Mynewt Wiki:
https://cwiki.apache.org/confluence/display/MYNEWT/Release+Notes

This release candidate was tested as follows:
1. Manual execution of the Mynewt test plan:
   
https://cwiki.apache.org/confluence/display/MYNEWT/Apache+Mynewt+Test+Plan
   The test results can be found at:
   https://cwiki.apache.org/confluence/display/MYNEWT/1.0.0-b1+Test+Results

2. The full unit test suite for this release was executed via "newt
   test all" with no failures.  This testing was performed on the
   following platforms:
 * OS X 10.10.5
 * Linux 4.4.6 (Gentoo)

The release candidate to be voted on is available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-1.0.0-b1-incubating/rc1/

The commits under consideration are as follows:

blinky:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-blinky
commit: b6918edc174b764f1e0b4ef4f4652aca8fedf8dd

core:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-core
commit: c6a8f88f6b8dc7c8eee6dc75a34c9b6ea295b05d

newt:
repos: https://git-wip-us.apache.org/repos/asf/incubator-mynewt-newt
commit: 59ec3af372952382390cfd988e503d8cf3d5b34b

In addition, the following newt convenience binaries are available:
linux: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-1.0.0-b1-incubating/rc1/apache-mynewt-newt-bin-linux-1.0.0-b1-incubating.tgz
osx: 
https://dist.apache.org/repos/dist/dev/incubator/mynewt/apache-mynewt-1.0.0-b1-incubating/rc1/apache-mynewt-newt-bin-osx-1.0.0-b1-incubating.tgz


The release candidate is signed with a GPG key available at:
https://dist.apache.org/repos/dist/dev/incubator/mynewt/KEYS

The vote is open for at least 72 hours and passes if a majority of at
least three +1 PPMC votes are cast.

[ ] +1 Release this package
[ ]  0 I don't feel strongly about it, but don't object
[ ] -1 Do not release this package because...

Anyone can participate in testing and voting, not just committers,
please feel free to try out the release candidate and provide your
votes.

A separate [DISCUSS] thread will be opened to talk about this release
candidate.

Thanks,
Chris

Re: Halting Mynewt

[Kevin Townsend]

The nRF51 also (strangely!) has a non standard SWD interface. It's the 
first time I've come across that, and I'm not sure what the thinking 
behind multiplexing RST with the SWD pins was (extremely low pin count 
devices?), but it breaks the normal behaviour of SWD on ARM Cortex M and 
you need to handle it as an edge case on the J-Link and other debuggers 
internally. Thankfully, the nRF52 doesn't have this same 'feature', and 
the peripherals are all around more pleasant to use. :)



On 22/11/16 18:23, Wayne Keenan wrote:

Do you have any other devices connected to your nRF (electronically, not
BLE) and are actively trying to send data to the nRF?
If so, detach them and try again.

In one repeatable case I've found it impossible to connect to an nRF51
using SWD via the J-Link when another MCU was communicating using SPI to
the nRF51.
This happened even if the nRF was totally erased, the SPI chatter just got
in the way of SWD working.

ymmv

All the best
Wayne

On 22 November 2016 at 17:07, David G. Simmons  wrote:


How are you keeping the board in reset? The only method I've found is to
power the board off, and so far, my timing has not paid off on that front.

dg


On Nov 22, 2016, at 12:04 PM, marko kiiskila  wrote:

Hi,

my trick with this is by keeping the board in reset just until I start

the

JTAG software. This might take few attempts, but I eventually get lucky
with timing, and MCU stops before system boots.

--
David G. Simmons
(919) 534-5099
Web  • Blog  •
Linkedin  • Twitter <
http://twitter.com/TechEvangelist1> • GitHub 
/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
  * http://www.gnupg.com/  Secure your email!!!
  * Public key available at keyserver.pgp.com 
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!

There are only 2 hard things in computer science: Cache invalidation,
naming things, and off-by-one errors.

Re: Halting Mynewt

[Kevin Townsend]

I generally put a factory reset pin in projects that I check very early 
on, which has saved my butt more than once, though it isn't fool proof. 
It's just nice to have an early stage option to go back to a known state 
when devices are in the field or even during development (and you can 
reformat the NFFS partition, etc., to kill any config settings).


Also ... if you have something like BOOT_SERIAL enabled in your 
bootloader, this also always gives you a safe option to recover a device 
since the bootloader should always be stable, and will execute before 
your app. Just enter serial bootloader, and you should have JLink access 
again.



On 22/11/16 18:18, David G. Simmons wrote:

NRF52DK. I managed to get it reset. Timing is hard to get perfect. The 'Reset' 
button is ignored.

I think this might point to the need for an os_suspend() or an os_stop() 
function.

dg


On Nov 22, 2016, at 12:13 PM, marko kiiskila  wrote:

Usually I have a button. But I have had to use jumper wires also :)
What board are you using?

--
David G. Simmons
(919) 534-5099
Web  • Blog  • Linkedin 
 • Twitter  • GitHub 

/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
  * http://www.gnupg.com/  Secure your email!!!
  * Public key available at keyserver.pgp.com 
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!

There are only 2 hard things in computer science: Cache invalidation, naming 
things, and off-by-one errors.

Re: nRF52DK BSP SPI Conflict?

[Kevin Townsend]

Oh good to know since I started working on an SPI driver today. I'll hold
off until the HAL changes are live then.

Le ven. 18 nov. 2016 à 20:49, will sanfilippo <wi...@runtime.io> a écrit :

> No, no need to do that as I am currently modifying the spi hal so that we
> dont have license issues with the nordic SDK. I will commit changes to the
> syscfg.yml files when I do this.
>
> Will
> > On Nov 18, 2016, at 4:09 AM, Kevin Townsend <ke...@adafruit.com> wrote:
> >
> > On 17/11/16 05:15, will sanfilippo wrote:
> >> There is an easy way to prevent this as well which should be added to
> syscfg.yml. The following works:
> >>
> >> SPI_0_MASTER:
> >> description: 'SPI 0 master'
> >> value:  1
> >> restrictions:
> >> - "!SPI_0_SLAVE"
> >> SPI_0_SLAVE:
> >> description: 'SPI 0 slave'
> >> value:  1
> >> restrictions:
> >> - "!SPI_0_MASTER"
> >> The newt tool generates a nice error message:
> >>
> >> Error: Syscfg restriction violations detected:
> >> SPI_0_MASTER=1 requires SPI_0_SLAVE not be set, but SPI_0_SLAVE=1
> >> SPI_0_SLAVE=1 requires SPI_0_MASTER not be set, but SPI_0_MASTER=1
> >>
> >> Setting history (newest -> oldest):
> >> SPI_0_MASTER: [hw/bsp/nrf52dk:1]
> >> SPI_0_SLAVE: [hw/bsp/nrf52dk:1]
> >
> > I wasn't aware of the restrictions flag, that's nice to know about.
> >
> > Should I submit a pull request to resolve the conflict in master, though
> (for at least the nRF52DK, I haven' t checked the other BSP packages)?
> >
> > I'd propose only enabling SPI_0_MASTER by default, but I'm perhaps
> missing some specific intentions with this as well (having both MASTER and
> SLAVE enabled in the BSP).
> >
> >
>
>

Re: nRF52DK BSP SPI Conflict?

[Kevin Townsend]

On 17/11/16 05:15, will sanfilippo wrote:

There is an easy way to prevent this as well which should be added to 
syscfg.yml. The following works:

 SPI_0_MASTER:
 description: 'SPI 0 master'
 value:  1
 restrictions:
 - "!SPI_0_SLAVE"
 SPI_0_SLAVE:
 description: 'SPI 0 slave'
 value:  1
 restrictions:
 - "!SPI_0_MASTER"
The newt tool generates a nice error message:

Error: Syscfg restriction violations detected:
 SPI_0_MASTER=1 requires SPI_0_SLAVE not be set, but SPI_0_SLAVE=1
 SPI_0_SLAVE=1 requires SPI_0_MASTER not be set, but SPI_0_MASTER=1

Setting history (newest -> oldest):
 SPI_0_MASTER: [hw/bsp/nrf52dk:1]
 SPI_0_SLAVE: [hw/bsp/nrf52dk:1]


I wasn't aware of the restrictions flag, that's nice to know about.

Should I submit a pull request to resolve the conflict in master, though 
(for at least the nRF52DK, I haven' t checked the other BSP packages)?


I'd propose only enabling SPI_0_MASTER by default, but I'm perhaps 
missing some specific intentions with this as well (having both MASTER 
and SLAVE enabled in the BSP).

Re: Shell Tutorial

[Kevin Townsend]

These are the notes I made on shell commands (attached). They should be 
up to date with master.


K.


On 17/11/16 19:45, Christopher Collins wrote:

Hi David,

On Thu, Nov 17, 2016 at 01:32:17PM -0500, David G. Simmons wrote:

I'm thinking of writing up a Tutorial on how to enable the Console and Shell in 
a Mynewt app -- especially since things have changed a bit with the Event Queue 
and task management.

If I am understanding things correctly (and there's no guarantee that I am) 
here's what needs to happen:

1) App needs to have an event Queue if it doesn't already
2) pkg.yml file needs to include
 - sys/console/full
 - sys/shell
3) syscfg.yml needs to set SHELL_TASK: 1

What else?

That looks correct to me.  The only other step that might be missing is:
enable specific shell commands with the appropriate syscfg settings
(e.g., OS_CLI, STATS_CLI).

Chris


# Shell Command Handler

## Adding shell support

To add shell support to your app make sure the following `pkg.deps` are
defined in your pkg.yml file:

```
pkg.deps:
- "@apache-mynewt-core/sys/console/full"
- "@apache-mynewt-core/sys/shell"
- "@apache-mynewt-core/sys/sysinit"
```

In the `syscfg.vals` section of syscfg.yml (>= 0.10.0) add the following:

```
syscfg.vals:
# Enable the shell task.
SHELL_TASK: 1
```

## Adding a default system event queue

In your main.c file add the following code snippets:

```
#include "sysinit/sysinit.h"
#include "console/console.h"
#include "shell/shell.h"

/* System event queue task handler */
#define SYSEVQ_PRIO (10)
#define SYSEVQ_STACK_SIZEOS_STACK_ALIGN(512)
static struct os_task task_sysevq;

/* Event queue for events handled by the system (shell, etc.) */
static struct os_eventq sys_evq;

/**
 * This task serves as a container for the shell and newtmgr packages.  These
 * packages enqueue timer events when they need this task to do work.
 */
static void
sysevq_handler(void *arg)
{
while (1) {
os_eventq_run(_evq);
}
}
```

In the `init_tasks` function (or in main if you prefer) add:

```
/**
 * init_tasks
 *
 * Called by main.c after sysinit(). This function performs initializations
 * that are required before tasks are running.
 *
 * @return int 0 success; error otherwise.
 */
static void
init_tasks(void)
{
os_stack_t *pstack;

/* Initialize eventq and designate it as the default.  Packages that need
 * to schedule work items will piggyback on this eventq.  Example packages
 * which do this are sys/shell and mgmt/newtmgr.
 */
os_eventq_init(_evq);

pstack = malloc(sizeof(os_stack_t)*SYSEVQ_STACK_SIZE);
assert(pstack);

os_task_init(_sysevq, "sysevq", sysevq_handler, NULL,
SYSEVQ_PRIO, OS_WAIT_FOREVER, pstack, SYSEVQ_STACK_SIZE);

/* Set the default eventq for packages that lack a dedicated task. */
os_eventq_dflt_set(_evq);
}
```

Between `sysinit` and `os_start` in main add:

```
init_tasks();
```

## Adding a custom command handler

To add a new command handler use the following code snippets:

```
// Command handler prototype declaration
static int shell_test_cmd(int argc, char **argv);

// Shell command struct
static struct shell_cmd shell_test_cmd_struct = {
.sc_cmd = "test",
.sc_cmd_func = shell_test_cmd
};

...

// Implement your command handler
static int
shell_test_cmd(int argc, char **argv)
{
console_printf("Test!\n");
return 0;
}

...

// Call this before os_init to register the command
#if MYNEWT_VAL(SHELL_TASK)
shell_cmd_register(_test_cmd_struct);
#endif
```

Re: Smoke testing latest release

[Kevin Townsend]

You'll need to change the chip ID (there is a list available on Segger's 
website), but if you are using a JLink you can also try this and then 
send the '*erase*' command once you're connected, just as a FYI:


   $ jlinkexe -device nrf52832_xxaa -if swd -speed auto -autoconnect 1


On 11/11/16 18:00, David G. Simmons wrote:

This was the bit I was missing ...

All good now! I guess I should add this to the tutorial.

dg


On Nov 11, 2016, at 11:55 AM, will sanfilippo  wrote:

# Erase the flash
newt debug arduino_zero_blinky

(gdb) mon at91samd chip-erase
chip erased
chip erased
(gdb) x/32wx 0
0x0:0x  0x  0x  0x
0x10:   0x  0x  0x  0x
0x20:   0x  0x  0x  0x
0x30:   0x  0x  0x  0x
0x40:   0x  0x  0x  0x
0x50:   0x  0x  0x  0x
0x60:   0x  0x  0x  0x
0x70:   0x  0x  0x  0x
(gdb) q

--
David G. Simmons
(919) 534-5099
Web  • Blog  • Linkedin 
 • Twitter  • GitHub 

/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
  * http://www.gnupg.com/  Secure your email!!!
  * Public key available at keyserver.pgp.com 
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!

There are only 2 hard things in computer science: Cache invalidation, naming 
things, and off-by-one errors.

Re: Sensor Drivers and File Location

[Kevin Townsend]

Raw data is still important in my opinion. 90% of the time a shares API 
for all accels or mags is the best choice, but there are instances where 
you will want the raw data for DSP filtering, etc., and I think there 
should be a mechanism to get standard SI as well as 'raw' values back 
from a device.



On 10/11/16 16:46, Sterling Hughes wrote:
I should also mention, another option that we could do is have drivers 
for the individual sensors (just have them expose native APIs as 
drivers, e.g. bmp2085_read_val()), but also map into a generic sensor 
API, that we locate in hw/sensors, as an example.


I’m not sure to what extent people using these sensors want raw access 
to them (i.e. this sensor has special features I care about), versus 
just treating them as a generic sensor.  If we think it will be common 
to have special features per-sensor, i’d say it makes sense to have:


hw/sensors — generic sensor API

and then

hw/drivers/sensors/

contain individual sensor drivers organized by manufacturer, that 
create their own device APIs, but also can register with the sensor 
API in hw/sensors.


Sterling

On 10 Nov 2016, at 15:42, Sterling Hughes wrote:


Hi Kevin,

I have some thoughts, but I’m not sure they all directly address your 
concerns :-)


More details on directory re-org and drivers below, but I wanted to 
mention upfront: I really think its worth thinking through the sensor 
api you mention in your PS, and then doing the directory org after 
that.  Because I would imagine that these drivers fit directly into 
that API, which I think is crucially important.  It would be really 
good to understand whether that API was sufficient for 99% of use 
cases as well, because if so, we would just want to have the 
implementations map into that API.


Additionally, I wanted to mention, that a worthy goal here would be 
to make these sensors discoverable, in a way that can be mapped into 
OIC.  One of the nice things about OIC is that it supports 
standardized resource discovery, and I think it would be awesome, 
especially for Maker applications, if you could connect to a device 
with a phone, and automatically get a list of sensors available, and 
interact with them.  If we make sure that the sensor APIs are 
introspectable, this should be easy to do.


I also want to point out how we’ve done certain drivers, like the ADC 
driver.  Essentially, we have a top-level package, “adc”, which 
contains the interface for all ADC drivers, and then we have the 
individual implementation of the driver itself.  The following source 
files point to this:


* high-level ADC API: hw/drivers/adc/include/adc/adc.h
- note: 2 interfaces defined here.  The driver implementation 
interface: adc_driver_funcs, and the user interface calls.


* implementation of ADC for NRF52: 
hw/drivers/adc/adc_nrf52/src/adc_nrf52.c
- note: only exported call is int nrf52_adc_dev_init(struct 
os_dev *, void *), which is passed as a function pointer to 
os_dev_create() in hal_bsp.c, when the adc device is created.


So the idea is that there is an API for the driver, and then 
sub-packages to that driver that can implement that API.  You have 
flexibility in the sub-packages, so you could do something like:


hw/drivers/sensors/src/sensors.c (*) main sensors driver
hw/drivers/sensors/pressure/bosch/bm32/src/bm32.c (*) bm32 
implementation.


Users would then create the device, with the bm32 sensor:

os_dev_create(“my_sensor”, bm32_init_func);

But can then open and use the device with the generic sensor API, 
without knowing the underlying implementation.


sensor = (struct sensor *) os_dev_open(“my_sensor”)
sensor_read(sensor, );

I assume you read most of this, but I just wanted to point this out.

As an aside: I certainly think we should use pkg.keywords extensively 
in the drivers directory, and make it easy to search for drivers.  We 
should also make sure that “newt pkg search”: a) works, and b) 
provides useful information about each of the drivers when there are 
matches.  There is going to be no _perfect_ way to organize this for 
ease of discovery, but I think newt can help a lot.


On 10 Nov 2016, at 1:59, Kevin Townsend wrote:

There are no sensor drivers in the system at present, but now that 
the HAL rework is complete I wanted to port a few drivers over to 
Mynewt just to properly test the HW and HAL out. Existing sensor 
drivers is one of the key factors to draw people into any embedded 
RTOS/system, so I think it's important to get this right to pull 
people into the Mynewt ecosystem.




+1 - I totally agree!

I'm curious what kind of organization would make the most sense 
moving forward, though. Assuming some drivers are included in the 
core mynewt repo instead of being in independent libraries, what 
would be an ideal root location to place them in the file structure?


- hw/drivers/sensors ?



Do we think that 90% of sensors will have a unified interface. I.e. 
sensors have discoverable channels, and those

Re: Sensor Drivers and File Location

[Kevin Townsend]

Hi Sterling,

I think this makes a lot of sense and the API is critical here. I 
definitely agree with the comments below that starting with the API is 
the right place. Being able to interrogate devices would also be very 
valuable, as well as some high level system to control sampling rates, 
power management, etc.


I really need to test out the I2C and SPI pins on our board so we can go 
into production, so I'll put a couple hacky drivers together for some 
common sensors for now just to play with the I2C and SPI HAL.


Let me know when you're back from London and I'll get you setup at 
Runtime with some common sensors since it's always useful to test this 
against real devices. I'm happy to contribute drivers, but if everyone 
has a set of a few common sensors it's nice to be able to make sure the 
ideas match reality.


I definitely have quite a few thoughts on what a sensor API should look 
like to maintain generic data across similar sensor types so that all 
accelerometers return standards SI units like m/s2, magnetometers return 
uTesla, etc., though I've never used OIC so I'll have to do some digging 
there first.


K.


On 10/11/16 16:42, Sterling Hughes wrote:

Hi Kevin,

I have some thoughts, but I’m not sure they all directly address your 
concerns :-)


More details on directory re-org and drivers below, but I wanted to 
mention upfront: I really think its worth thinking through the sensor 
api you mention in your PS, and then doing the directory org after 
that.  Because I would imagine that these drivers fit directly into 
that API, which I think is crucially important.  It would be really 
good to understand whether that API was sufficient for 99% of use 
cases as well, because if so, we would just want to have the 
implementations map into that API.


Additionally, I wanted to mention, that a worthy goal here would be to 
make these sensors discoverable, in a way that can be mapped into 
OIC.  One of the nice things about OIC is that it supports 
standardized resource discovery, and I think it would be awesome, 
especially for Maker applications, if you could connect to a device 
with a phone, and automatically get a list of sensors available, and 
interact with them.  If we make sure that the sensor APIs are 
introspectable, this should be easy to do.


I also want to point out how we’ve done certain drivers, like the ADC 
driver.  Essentially, we have a top-level package, “adc”, which 
contains the interface for all ADC drivers, and then we have the 
individual implementation of the driver itself.  The following source 
files point to this:


* high-level ADC API: hw/drivers/adc/include/adc/adc.h
- note: 2 interfaces defined here.  The driver implementation 
interface: adc_driver_funcs, and the user interface calls.


* implementation of ADC for NRF52: 
hw/drivers/adc/adc_nrf52/src/adc_nrf52.c
- note: only exported call is int nrf52_adc_dev_init(struct os_dev 
*, void *), which is passed as a function pointer to os_dev_create() 
in hal_bsp.c, when the adc device is created.


So the idea is that there is an API for the driver, and then 
sub-packages to that driver that can implement that API.  You have 
flexibility in the sub-packages, so you could do something like:


hw/drivers/sensors/src/sensors.c (*) main sensors driver
hw/drivers/sensors/pressure/bosch/bm32/src/bm32.c (*) bm32 
implementation.


Users would then create the device, with the bm32 sensor:

os_dev_create(“my_sensor”, bm32_init_func);

But can then open and use the device with the generic sensor API, 
without knowing the underlying implementation.


sensor = (struct sensor *) os_dev_open(“my_sensor”)
sensor_read(sensor, );

I assume you read most of this, but I just wanted to point this out.

As an aside: I certainly think we should use pkg.keywords extensively 
in the drivers directory, and make it easy to search for drivers.  We 
should also make sure that “newt pkg search”: a) works, and b) 
provides useful information about each of the drivers when there are 
matches.  There is going to be no _perfect_ way to organize this for 
ease of discovery, but I think newt can help a lot.


On 10 Nov 2016, at 1:59, Kevin Townsend wrote:

There are no sensor drivers in the system at present, but now that 
the HAL rework is complete I wanted to port a few drivers over to 
Mynewt just to properly test the HW and HAL out. Existing sensor 
drivers is one of the key factors to draw people into any embedded 
RTOS/system, so I think it's important to get this right to pull 
people into the Mynewt ecosystem.




+1 - I totally agree!

I'm curious what kind of organization would make the most sense 
moving forward, though. Assuming some drivers are included in the 
core mynewt repo instead of being in independent libraries, what 
would be an ideal root location to place them in the file structure?


- hw/drivers/sensors ?



Do we think that 90% of sensors will have a unified interface. I.e. 
sensors have discoverable

Re: [incubator-mynewt-blinky] Git Push Summary

[Kevin Townsend]

On 10/11/16 13:23, Sterling Hughes wrote:

Is the idea to merge these from develop or master?

I think we should probably merge develop->master now, prior to 
branching anything.  We should only be branching releases off of 
master (which I’m assuming is the intention.)


+1 for this. :) It's difficult to do multi person development today 
against develop since everyone needs to ensure they are on the same 
commit, but there are so many important changes in develop that working 
against master isn't viable today. That said, all the changes so far 
have been worth the trouble and it will never be easier to break things 
so if it takes more time it's better to drag it out a bit longer.

Re: Sensor Drivers and File Location

[Kevin Townsend]

This avoids the problem of a sensor that is has an accelerometer AND a 
magnetometer as they will typically both use the same bus. In the case of 
sensors that can use, say, I2C or SPI, you'd need 2 drivers anyway, so rather 
than having /hw/drivers/sensors/bno055/spi/ and /hw/drivers/sensors/bno055/I2C 
you'd have a bno055 driver in both the /hw/drivers/SPI and /hw/drivers/I2C 
areas.

Missed this when I replied on my phone, sorry.

I can see the argument here, but to be honest I would rather maintain 1 
driver that support both I2C and SPI simply because there is still a lot 
of overlapping code above the transport. Registers, data conditioning, 
etc., and there is a non neglible risk with two drivers that bugs get 
fixed in one driver but not the other.


That said ... there isn't a clear right answer here. I'm worried about 
what happens when you have 40-50 drivers in a flat folder, which is 
entirely realistic over time since embedded generally = sensors, 
although that still feels like the only solution that doesn't run into 
problems quickly separating files by functionality or bus.


hw/drivers/sensors// will reduce this to a 
certain extent.


Another question is what about chips that aren't technically sensors:

 * Motor drivers
 * Display drivers or integrated displays (SSD1306, ILI9341, etc.)

These are all commonly used in embedded systems.


Perhaps a parent`ic` folder is a better label than 'sensors', so:

 * hw/drivers/ic/display/ssd1306
 * hw/drivers/ic/sensors/tls2561
 * hw/drivers/ic/sensors/bno055
 * hw/drivers/ic/motor/drv2605l

You don't want to end up with something 10 layers deep, but I think it 
is important to keep this clean and well organised since it is bound to 
be a key part of the system moving forward, although I expect a lot of 
drivers will be stored outside the core repo of course.


K.

Re: Sensor Drivers and File Location

[Kevin Townsend]

Many sensors support both i2c and SPI though which might pose a problem or
oblige you to have two drivers, one for each bus?

Le jeudi 10 novembre 2016, David G. Simmons <santa...@mac.com> a écrit :

> I would actually go about this slightly differently ...
>
> All of these sensors come in, typically, one flavor. So there are
> SPI-based sensors, and ADC-based sensors, etc. Grouping them by the bus
> they use makes a certain amount of sense to me.
>
> Hw/drivers/SPI/...
> hw/drivers/ADC/...
> ...
>
> That way I know that if I have a SPI-based sensor, I know exactly where to
> look for the driver for that sensor.
>
> This may also come in handy when we look at the other side of peripherals,
> actuators.
>
> hw/drivers/servo/...
> hw/drivers/pwm/...
> hw/drivers/stepper/...
>
> This avoids the problem of a sensor that is has an accelerometer AND a
> magnetometer as they will typically both use the same bus. In the case of
> sensors that can use, say, I2C or SPI, you'd need 2 drivers anyway, so
> rather than having /hw/drivers/sensors/bno055/spi/ and
> /hw/drivers/sensors/bno055/I2C you'd have a bno055 driver in both the
> /hw/drivers/SPI and /hw/drivers/I2C areas.
>
> Hope that makes sense.
>
> dg
>
> > On Nov 9, 2016, at 8:59 PM, Kevin Townsend <ke...@adafruit.com
> <javascript:;>> wrote:
> >
> > There are no sensor drivers in the system at present, but now that the
> HAL rework is complete I wanted to port a few drivers over to Mynewt just
> to properly test the HW and HAL out. Existing sensor drivers is one of the
> key factors to draw people into any embedded RTOS/system, so I think it's
> important to get this right to pull people into the Mynewt ecosystem.
> >
> > I'm curious what kind of organization would make the most sense moving
> forward, though. Assuming some drivers are included in the core mynewt repo
> instead of being in independent libraries, what would be an ideal root
> location to place them in the file structure?
> >
> > - hw/drivers/sensors ?
> >
> > The problem is that hw/drivers already has things like adc and uart
> implementations or nimble which is quite a different concept, though it's
> not entirely inappropriate all the same.
> >
> > And inside sensors (or wherever) you'll have another problem: should you
> have a single flat list of all drivers by device name, or do you want to
> organize them by type/manufacturer/etc.:
> >
> > * hw/drivers/sensors/bmp085/*
> > * hw/drivers/sensors/bmp280/*
> > * hw/drivers/sensors/tsl2651/*
> > * hw/drivers/sensors/lsm303dlhc/*
> > * hw/drivers/sensors/bno055/*
> >
> > Or do you want to organize them by family (which is nice in some ways
> but very awkward in others with certain SoCs):
> >
> > * hw/drivers/sensors/pressure/bmp085/*
> > * hw/drivers/sensors/pressure/bmp280/*
> > * hw/drivers/sensors/light/tsl2561/*
> > * hw/drivers/sensors/accelerometers/lsm303dlhc/* <-- This is also a
> >   magnetometer!
> > * hw/drivers/sensors/orientation???/bno055/* <-- this is an accel, mag
> >   and gyro with sensor fusion for orientation
> >
> > Or does manufacturer make more sense:
> >
> > * hw/drivers/sensors/bosch/bmp085/*
> > * hw/drivers/sensors/bosch/bmp280/*
> > * hw/drivers/sensors/bosch/bno055/*
> > * hw/drivers/sensors/taos/tsl2561/*   <-- Complicated since Taos was
> >   purchased and is now AMS, which is a recurring theme these days
> > * hw/drivers/sensors/st/lsm303dlhc/*
> >
> > It would likely be useful to have some sort of basic meta-data as well
> about the sensor types since some ICs can contain multiple sensors, such as
> the bmp280 being a pressure sensor but also having temperature data, or the
> lsm303dlhc being a 3 axis accelerometer and magnetometer. This meta-data
> isn't critical, but could be useful as a filter at some point using the
> newt tool or the newt 'newt vals' type functionality.
> >
> > Adding drivers to the core repo is problematic in that it creates a
> maintenance burden, but I think drivers are also a huge pull for people to
> use the system and an important reference to keep up to date with changes
> in the HAL over time. There should at least be one reference for each bus
> like I2C, SPI, ADC and UART (GPS etc.) that makes use of the power
> management API, etc. Having a maintained reference driver will reduce
> support requirements long term, and ensure best practices are followed by
> people contributing other drivers in the future.
> >
> > Kevin
> >
> > PS: Some sort of sensor API is also worth considering to have a common
> data type and standard SI 

Sensor Drivers and File Location

[Kevin Townsend]

There are no sensor drivers in the system at present, but now that the 
HAL rework is complete I wanted to port a few drivers over to Mynewt 
just to properly test the HW and HAL out. Existing sensor drivers is one 
of the key factors to draw people into any embedded RTOS/system, so I 
think it's important to get this right to pull people into the Mynewt 
ecosystem.


I'm curious what kind of organization would make the most sense moving 
forward, though. Assuming some drivers are included in the core mynewt 
repo instead of being in independent libraries, what would be an ideal 
root location to place them in the file structure?


- hw/drivers/sensors ?

The problem is that hw/drivers already has things like adc and uart 
implementations or nimble which is quite a different concept, though 
it's not entirely inappropriate all the same.


And inside sensors (or wherever) you'll have another problem: should you 
have a single flat list of all drivers by device name, or do you want to 
organize them by type/manufacturer/etc.:


 * hw/drivers/sensors/bmp085/*
 * hw/drivers/sensors/bmp280/*
 * hw/drivers/sensors/tsl2651/*
 * hw/drivers/sensors/lsm303dlhc/*
 * hw/drivers/sensors/bno055/*

Or do you want to organize them by family (which is nice in some ways 
but very awkward in others with certain SoCs):


 * hw/drivers/sensors/pressure/bmp085/*
 * hw/drivers/sensors/pressure/bmp280/*
 * hw/drivers/sensors/light/tsl2561/*
 * hw/drivers/sensors/accelerometers/lsm303dlhc/* <-- This is also a
   magnetometer!
 * hw/drivers/sensors/orientation???/bno055/* <-- this is an accel, mag
   and gyro with sensor fusion for orientation

Or does manufacturer make more sense:

 * hw/drivers/sensors/bosch/bmp085/*
 * hw/drivers/sensors/bosch/bmp280/*
 * hw/drivers/sensors/bosch/bno055/*
 * hw/drivers/sensors/taos/tsl2561/*   <-- Complicated since Taos was
   purchased and is now AMS, which is a recurring theme these days
 * hw/drivers/sensors/st/lsm303dlhc/*

It would likely be useful to have some sort of basic meta-data as well 
about the sensor types since some ICs can contain multiple sensors, such 
as the bmp280 being a pressure sensor but also having temperature data, 
or the lsm303dlhc being a 3 axis accelerometer and magnetometer. This 
meta-data isn't critical, but could be useful as a filter at some point 
using the newt tool or the newt 'newt vals' type functionality.


Adding drivers to the core repo is problematic in that it creates a 
maintenance burden, but I think drivers are also a huge pull for people 
to use the system and an important reference to keep up to date with 
changes in the HAL over time. There should at least be one reference for 
each bus like I2C, SPI, ADC and UART (GPS etc.) that makes use of the 
power management API, etc. Having a maintained reference driver will 
reduce support requirements long term, and ensure best practices are 
followed by people contributing other drivers in the future.


Kevin

PS: Some sort of sensor API is also worth considering to have a common 
data type and standard SI units and meta-data for all sensors (min/max 
data values, output speed, etc.), but that's another discussion entirely.

Re: Nordic SDK license not Apache-compatible?

[Kevin Townsend]

We have a similar problem with Nordic code where due to older 
restrictive license terms we can't release the source for some projects. 
The terms have gotten friendlier with time, but the earliest stuff was 
quite restrictive.


I'd suggest getting in touch with Nordic, though. They're probably open 
to making some changes in certain situations. If you need a contact 
there, just send me a private email but I know Sterling already knows a 
mutual contact that would be a good person to explain the issue to and 
hopefully get the ball rolling on what will ideally lead to compatible 
license terms(???). It seems like it would be in their interest and most 
of this code isn't useful on anything other than their own silicon anyway.



On 10/11/16 02:12, Christopher Collins wrote:

Hi Justin,

On Thu, Nov 10, 2016 at 08:33:54AM +1100, Justin Mclean wrote:

Hi,


The first clause, Grant of License, seems to be problematic:

Look like it "non-sub licensable” may be an issue? And "solely in
connection with a Nordic Integrated Circuit” reads like a field of use
restriction to me [1]

Section 4 of the license re distribution may also be a concern.

That was indeed my fear.  Thanks for calling attention to section 4 as
well.


Their modified BSD license also may be an issue as it also looks to
includes a field of use restriction.

You are right - I hadn't noticed that.  However, the fourth clause
is missing in the actual commits that were made to address this ticket
(https://github.com/ARMmbed/ble-nrf51822/commit/6d1bf116e156b870099694f0ce27076c236c4f44).
  Maybe there was some additional communication between the mbed team and 
Nordic that let to the actual changes.


I assume this is optional in that not everyone would need to use it?

Yes, these files are optional in the sense that Mynewt can still be
used without them.  However, someone wishing to use Mynewt with a Nordic
MCU will probably won't get very far without them.


1. https://www.apache.org/legal/resolved#category-x

Thanks, Chris

ISO8601 Timestamps

[Kevin Townsend]

Would it make sense to add support to datetime.c/h to convert to and 
from ISO8601 timestamps, which are much more convenient to display since 
you can use a struct like this to print the values directly in the 
shell, etc.:


   typedef struct ATTR_PACKED
   {
char year[4];/**< Year */
char dash1;  /**< Dash1*/
char month[2];   /**< Month*/
char dash2;  /**< Dash2*/
char day[2]; /**< Day  */
char T;  /**< T*/
char hour[2];/**< Hour */
char colon1; /**< Colon1   */
char minute[2];  /**< Minute   */
char colon2; /**< Colon2   */
char second[2];  /**< Second   */
char decimal;/**< Decimal  */
char sub_second[6];  /**< Sub-second   */
char Z;  /**< UTC timezone */

char nullterm;   // not part of the format, make printf easier
   } iso8601_time_t;

* Ignore the typedef etc. which breaks the mynewt rules, this is copied 
from another project.


Maybe something like *_to_iso8601 and iso8601_to_*, converting between 
epoch and iso8601 etc.


This just has the advantage of being able to print human readable 
timestamps in a well-defined format: https://en.wikipedia.org/wiki/ISO_8601


The way the typedef is formatted above means you can just print data 
directly and get something like this: *2016-11-08T07:24:30.123456Z* 
which can in turn be understand by most other systems.


Just curious if this adds enough value to be included or if epoch is 
generally preferable as the sole time keeping units in the core codebase.


K.

Re: How to link the thirdparty library such as .a file into myproject?

[Kevin Townsend]

Hi,

I don’t think that is possible at the moment. That needs to be added,
as the unfortunate reality is that too often we end up getting binary
drops of libraries :(
A big +1 for this ... sadly, most graphics libraries, complex protocol 
stacks and a lot of security code is delivered in binary blobs today, 
and there aren't always viable open source alternatives to these kind of 
problems in the real world on normal deadlines and budgets. :(

Re: newt and newtmgr versions

[Kevin Townsend]

Hi Sterling,
I'm just migrating over to develop for mynewt-core and the newt tools 
repos, and noticed the following after updating the command line 
tools from develop:


- 'newt version' -> returns 0.9.0 in develop, this should probably be 
0.10.0 to verify that you are using something different than the 
master branch?


+1
I submitted a pull request ticking this up on (super trivial but at 
least it's easy to integrate).  I'm not sure if NewtBlinkyTag is 
correct, though? Feel free to edit or reject if so: 
https://github.com/apache/incubator-mynewt-newt/pull/25
We don’t really have a policy here, which is why the newt version 
hasn’t been updated.  I think once we roll a release, we should always 
update newt to the subsequent version, so that people can 
differentiate.  We might want to have some indicator that it is built 
off a non-released version as well (maybe a build date?)


If the version fields have to be integer values then perhaps in 
'develop' releases you can insert the 32-bit unix epoch timestamp as the 
last field? It isn't very human readable, but it does at least indicate 
a clear difference between updates. It seems like this would fit:


struct image_version {
uint8_t iv_major;
uint8_t iv_minor;
uint16_t iv_revision;
uint32_t iv_build_num;
};

That said, it isn't a very elegant solution.

Another alternative might be a magic number that is converted to '-dev' 
when detected. You can convert '-dev' to it's char equivalent in 
uint32_t, for example, since it's 4 bytes wide.

Re: i2c hal API issues and possible modifications

[Kevin Townsend]

Hi Will,

Having control over when the STOP happens is important since different 
devices will behave differently, especially if you are performing a 
series of events like read/write/read or something similar. Some devices 
expect the STOP to happen in different places in a command sequence.


My vote would be a less attractive AP where the stop is a flag. Since we 
apparently aren't using _begin to set the START we may as well drop the 
STOP being handled in _end, but I'm obviously curious what other people 
think here.


Sending multiple STARTS is perfectly valid BTW: 
http://www.i2c-bus.org/repeated-start-condition/


K.

On 18/10/16 21:58, will sanfilippo wrote:

Hello:

I am new to i2c so please bear with me. Hopefully this makes sense to the i2c 
experts out there.

In adding i2c to the nordic platforms I ran across an issue that I am not sure 
how to solve. Well, I know one way to solve it but it would require changes to 
the API. Here is the issue:

In order to get the nordic HW to generate a NACK bit (well, the opposite of an 
ACK) on the 9th bit after a read, you need to do something special to the HW 
prior to reading the last received character. When you do this, the HW will 
generate the correct 9th bit and will also put a STOP condition on the line. 
While it is possible to get the HW to generate a STOP independently, I dont see 
how I can get it to generate the correct 9th bit unless you know it is the last 
character you want to read.

Another issue that is bothersome is the nordic SDK itself: when you are done 
with the read it automatically generates the STOP. That is fine in and of 
itself; the SDK could be modified, but the issue is that dang 9th bit.

Our API has the following functions:
hal_i2c_master_begin(): Not exactly sure what this does on all platforms but 
currently is a NOOP on the nordic platforms.
hal_i2c_master_write(): generates a start but no stop.
hal_i2c_master_read(): generates a start but no stop.
hal_i2c_master_end(): generates a stop

At first glance, this seems to be a fine API and is easy to see a 
“transaction”: a begin, writes and/or reads chained together, and an end. 
Unfortunately, I could not see how to do this on the nordic platforms.

One way to do this would be to get rid of begin/end and have a flag in the 
read/write API. The flag would be “generate stop”. This way users can chain 
reads/writes together and end them with a STOP whenever they want. The only con 
to this is that it is not so easy to look at the code to see that transactions 
have a stop.

Given that I dont have alot of experience wth many i2c devices, I dont know if 
having the ability to skip START and/or address when the read/write APIs are 
called is useful. So far in my limited experience the start/address condition 
between the read/writes is not an issue.

So to summarize, here is what I suggest:
* Remove the begin/end API.
* Add a flag to write/read that will cause a STOP to be generated at the end of 
the read write.

Thanks! Comments greatly appreciated.

Re: Updating newt

[Kevin Townsend]

I haven't tried 0.10.0 but it's probably in the source.

It seems to be defined here so you can look at that file: 
https://github.com/apache/incubator-mynewt-newtmgr/blob/develop/newt/newtutil/newtutil.go


K.

On 07/10/16 16:22, David G. Simmons wrote:

Outstanding! Thanks. Am I to understand then that the version number has not 
changed and should still report 0.9.0 after this? I would think that it would 
give me 0.10.0

% git branch
* develop
   master
% git checkout
Your branch is up-to-date with 'origin/0_10_0_dev
% go install
% newt
Apache Newt (incubating) version: 0.9.0

dg


On Oct 7, 2016, at 10:10 AM, Kevin Townsend <ke...@adafruit.com> wrote:

Hi David,

These are the notes I made for myself on updating newt and newtmgr:

# Updating `newt` and `newtmgr`

To update the `newt` and `newtmgr` tools:

- Go to `$GOPATH/src/mynewt.apache.org/newt/`
- Run `git pull` to get the latest source code for the tools
- Switch branches if necessary (ex. `git checkout develop`)
- Go into the `newtmgr` sub-folder
- Run `go install`
- Repeat the process from the `newt` and `newtvm` folders

This will cause the tools in $GOPATH/bin to be rebuilt, which you can see
by running `ls -l $GOPATH/bin`

K.


On 07/10/16 16:07, David G. Simmons wrote:

Hi all,

I know there must be an easy way to do this, but I haven't found it yet.

I'm trying to get newt up to the latest version in the develop branch.  Here's 
what I've done so far:

% cd /Users/dsimmons/dev/go/src/mynewt.apache.org/newt 
<http://mynewt.apache.org/newt>
% newt version
Apache Newt (incubating) version: 0.9.0
% git branch --set-upstream-to=origin/0_10_0_dev
Branch develop set up to track remote branch 0_10_0_dev from origin.
% git rebase
Fast-forwarded develop to refs/remotes/origin/0_10_0_dev.
% git pull
Already up-to-date.

That last part doesn't seem right to me.

% ./build.sh
Building newt.  This may take a minute...
Successfully built executable: 
/Users/dsimmons/dev/go/src/mynewt.apache.org/newt/newt/newt 
<http://mynewt.apache.org/newt/newt/newt>
$ /Users/dsimmons/dev/go/src/mynewt.apache.org/newt/newt/newt 
<http://mynewt.apache.org/newt/newt/newt> version
Apache Newt (incubating) version: 0.9.0

And that part definitely isn't right.

If I can get some help in how to actually update newt, then I'll write up a doc 
on how to do it for others.

dg
--
David G. Simmons
(919) 534-5099
Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin 
<http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub 
<http://github.com/davidgs>
/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
  * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
  * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!

There are only 2 hard things in computer science: Cache invalidation, naming 
things, and off-by-one errors.




--
David G. Simmons
(919) 534-5099
Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin 
<http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub 
<http://github.com/davidgs>
/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
  * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
  * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!

There are only 2 hard things in computer science: Cache invalidation, naming 
things, and off-by-one errors.

Re: Updating newt

[Kevin Townsend]

Hi David,

These are the notes I made for myself on updating newt and newtmgr:

# Updating `newt` and `newtmgr`

To update the `newt` and `newtmgr` tools:

- Go to `$GOPATH/src/mynewt.apache.org/newt/`
- Run `git pull` to get the latest source code for the tools
- Switch branches if necessary (ex. `git checkout develop`)
- Go into the `newtmgr` sub-folder
- Run `go install`
- Repeat the process from the `newt` and `newtvm` folders

This will cause the tools in $GOPATH/bin to be rebuilt, which you can see
by running `ls -l $GOPATH/bin`

K.


On 07/10/16 16:07, David G. Simmons wrote:

Hi all,

I know there must be an easy way to do this, but I haven't found it yet.

I'm trying to get newt up to the latest version in the develop branch.  Here's 
what I've done so far:

% cd /Users/dsimmons/dev/go/src/mynewt.apache.org/newt 

% newt version
Apache Newt (incubating) version: 0.9.0
% git branch --set-upstream-to=origin/0_10_0_dev
Branch develop set up to track remote branch 0_10_0_dev from origin.
% git rebase
Fast-forwarded develop to refs/remotes/origin/0_10_0_dev.
% git pull
Already up-to-date.

That last part doesn't seem right to me.

% ./build.sh
Building newt.  This may take a minute...
Successfully built executable: 
/Users/dsimmons/dev/go/src/mynewt.apache.org/newt/newt/newt 

$ /Users/dsimmons/dev/go/src/mynewt.apache.org/newt/newt/newt 
 version
Apache Newt (incubating) version: 0.9.0

And that part definitely isn't right.

If I can get some help in how to actually update newt, then I'll write up a doc 
on how to do it for others.

dg
--
David G. Simmons
(919) 534-5099
Web  • Blog  • Linkedin 
 • Twitter  • GitHub 

/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
  * http://www.gnupg.com/  Secure your email!!!
  * Public key available at keyserver.pgp.com 
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!

There are only 2 hard things in computer science: Cache invalidation, naming 
things, and off-by-one errors.

SystemView with Mynewt

[Kevin Townsend]

Has anyone tried to get Segger's SystemView working with Mynewt? 
https://www.segger.com/systemview.html


I suspect most nRF52 devs are using a JLink, and live task tracking 
would be a nice compliment to the stats modules.


I've been meaning to try it out, I just wanted to see if any attempts 
had already been made at wiring the RTOS up to the SystemView API already.

Re: Unsolicited Notify with newtmgr over BLE

[Kevin Townsend]

I think you're right about that - a CCCD should not be set to one unless
the peer writes to it.  What I'm not so sure about is whether it is
prohibited to send a notification to an unsubscribed peer.  I didn't see
any language in the spec indicating that this is illegal.  The ability
to send unsolicited notifications is useful, and I don't see a reason
why it shouldn't be allowed.

I agree it's somewhat ambiguous or open to interpretation at present,
but in the case of something as critical as newtmgr over ble (OTA DFU
etc.), it probably makes sense to take the safe approach and add the
CCCD to a characteristic and have the peer explicitly set the notify
bit before the response data is sent???

Having the possibility to send unsolicited notifies has some merit, and
personally I don't have any objections to it being included as an option
in nimble, but I wouldn't rely on those assumptions for newtmgr over
BLE.

Just my opinion though and curious to hear what other people think.

Re: Improve Unit Tests and Test Suite Output

[Kevin Townsend]

Sorry, to answer my own question the 'test' app in apache-mynewt-core 
show how tests can be run on native HW.



On 06/10/16 00:16, Kevin Townsend wrote:

Hi Sterling,

Are you able to run the unit tests on real HW via newt to pipe the 
results back to the console? That would probably remove the need to 
'mock' peripherals in most cases, and be a significantly easier way to 
run a set of tests that have specific HW requirements, such as BLE 
which isn't available in the simulator today (understandably). Or are 
the unit tests currently limited to the simulator running as a native 
binary? The latter was my understanding but I haven't dug very deeply 
into it either.


Using the native BLE HW on OS X or via Bluez with the simulator would 
of course be /amazing/, but I think there are a WHOLE LOT of other 
higher priority features to add before that. :)


K.

Re: Unsolicited Notify with newtmgr over BLE

[Kevin Townsend]

Also, would it make sense to have more consistent UUIDs, such as:

- 8D530001-1DB7-4CD3-868B-8A527460AA84
- 8D530002-1DB7-4CD3-868B-8A527460AA84
- 8D530003-1DB7-4CD3-868B-8A527460AA84 (potentially?)

We could perhaps even settle on a single base 128-bit UUID throughout 
Mynewt, adjusting the 16 bit range at the start?


This is a minor detail and might be a nuisance to change, and more of a 
cosmetic issue, but I figured it was worth tossing out there.



On 06/10/16 02:00, Kevin Townsend wrote:
I'm working with newtmgr over BLE based on the newtmgr lib in the 
master branch of apache-mynewt-core, which currently has the following 
structure:


Service UUID: 8D53DC1D-1DB7-4CD3-868B-8A527460AA84
Characteristic UUID: DA2E7828-FBCE-4E01-AE9E-261174997C48

The source code has this explanatory comment on the service:

> The "newtmgr" service consists of one write no-rsp characteristic for
> newtmgr requests: a single-byte characteristic that can only accepts
> write-without-response commands. The contents of each write command
> contains an NMP request. NMP responses are sent back in the form of
> unsolicited notifications from the same characteristic.

I remember some discussions previously about unsolicited notifies and 
whether this was or wasn't supported in the BLE spec (I believe 
between Carles Cufi from Nordic and I believe Will and/or Chris at 
Runtime?), but given the uncertainty around this I wonder if it 
wouldn't be better to adjust the newtmgr service definition to have 
two characteristics similar to the way 'nus' (Nordic UART Service) works:


- One TXD or 'Request' characteristic to write commands to, which can 
reuse the existing characteristic as-is
- A new RXD or 'Response' characteristic with notify enabled, which 
would also allow you to send multiple data chunks since each packet 
will cause a new notify event on the Central, making it relatively 
easy to manage rebuilding packets.


I've been doing some initial tests with the netmgr service as is and 
many apps don't seem to like the unsolicited notifications even if 
they can be made to work.  Two separate characteristics would probably 
ensure everything is handled unambiguously on any platform, but I'm 
curious to hear other people's thoughts?


Kevin

Unsolicited Notify with newtmgr over BLE

[Kevin Townsend]

I'm working with newtmgr over BLE based on the newtmgr lib in the master 
branch of apache-mynewt-core, which currently has the following structure:


Service UUID: 8D53DC1D-1DB7-4CD3-868B-8A527460AA84
Characteristic UUID: DA2E7828-FBCE-4E01-AE9E-261174997C48

The source code has this explanatory comment on the service:

> The "newtmgr" service consists of one write no-rsp characteristic for
> newtmgr requests: a single-byte characteristic that can only accepts
> write-without-response commands. The contents of each write command
> contains an NMP request. NMP responses are sent back in the form of
> unsolicited notifications from the same characteristic.

I remember some discussions previously about unsolicited notifies and 
whether this was or wasn't supported in the BLE spec (I believe between 
Carles Cufi from Nordic and I believe Will and/or Chris at Runtime?), 
but given the uncertainty around this I wonder if it wouldn't be better 
to adjust the newtmgr service definition to have two characteristics 
similar to the way 'nus' (Nordic UART Service) works:


- One TXD or 'Request' characteristic to write commands to, which can 
reuse the existing characteristic as-is
- A new RXD or 'Response' characteristic with notify enabled, which 
would also allow you to send multiple data chunks since each packet will 
cause a new notify event on the Central, making it relatively easy to 
manage rebuilding packets.


I've been doing some initial tests with the netmgr service as is and 
many apps don't seem to like the unsolicited notifications even if they 
can be made to work.  Two separate characteristics would probably ensure 
everything is handled unambiguously on any platform, but I'm curious to 
hear other people's thoughts?


Kevin

Re: Improve Unit Tests and Test Suite Output

[Kevin Townsend]

Hi Sterling,

Are you able to run the unit tests on real HW via newt to pipe the 
results back to the console? That would probably remove the need to 
'mock' peripherals in most cases, and be a significantly easier way to 
run a set of tests that have specific HW requirements, such as BLE which 
isn't available in the simulator today (understandably). Or are the unit 
tests currently limited to the simulator running as a native binary? The 
latter was my understanding but I haven't dug very deeply into it either.


Using the native BLE HW on OS X or via Bluez with the simulator would of 
course be /amazing/, but I think there are a WHOLE LOT of other higher 
priority features to add before that. :)


K.


On 05/10/16 19:28, Sterling Hughes wrote:

Hi,

I don’t think we planned on providing a mock’ing framework in V1 of 
Mynewt.  The approach to mocking has been to implement the lower 
layers on sim, and then special case things where it only makes sense 
for a particular regression or unit test.  While you won’t get the 
control you have with mocking (i.e. guaranteed set of responses to 
external function calls), it does allow for a fair number of 
regression tests to run simulated — and should catch the vast majority 
of cases.


Going forward, it does sound like having this ability would be 
useful.  If somebody wanted to provide a patch to newt, that allows it 
to either use an external framework like CMock, or generate a set of 
mock templates itself, I think it would be a great contribution!


Sterling

On 3 Oct 2016, at 12:26, hathach wrote:


Hi all,

I previously used CMock & Unity as unit testing framework for my own 
project. CMock is rather complex since it allows mocking the lower 
layers thus isolating module and making it easy for testing/probing 
its behavior.


For example, when testing an service-adding function, all we care 
about is the ble_gatts_register_svcs()  finally invoked with the 
exact same svc_def. Behavior of ble_gatts_register_svcs() is subject 
to its own unit testing.


Though newt's testutil is still in developing stage, do we have a 
plan to implement some level of mocking framework like CMock. Since 
it will be a challenge to simulate lower layer and stimulate some 
certain scenario.


PS: I found even with mocking, it is also hard to do decent coverage 
of unit test with stuffs like peripherals. And the integration test 
is completely out of control :(


On 03/10/2016 22:42, Christopher Collins wrote:

Hi Kevin,

On Mon, Oct 03, 2016 at 03:08:32PM +0200, Kevin Townsend wrote:

I was wondering if there were any suggestions on how it might be
possible to improve the output of the unit tests to be a bit more
verbose, following some of the other frameworks out there for embedded
systems like CMOCK.

Unit testing and test simulation is an important part of the system 
for

any professionally maintained project, but could be even more useful
with a little bit of refinement.

Personally, I find it useful to see a list of tests being run and 
maybe
spot if I missed a module, and to know how many tests were run, 
etc., so

something like this when running the test suite(s)?

 Running 'TestSuiteName' test suite:
Running 'Test Name' ... [OK][FAILED]
Running 'Test Name' ... [OK][FAILED]
[n] unit tests passed, [n] failed

 Running 'TestSuiteName' test suite:
Running 'Test Name' ... [OK][FAILED]
Running 'Test Name' ... [OK][FAILED]
[n] unit tests passed, [n] failed

 Ran [n] unit tests in [n] test suites
 [n] unit tests passed, [n] failed

It's a poor example that needs more thought, but I was interested in
getting the discussion started.

The thinking was that the user doesn't want to be bothered with a bunch
of text when there are no failures.  That said, I agree that more
verbose output in the success case would be useful in some cases.  You
can get something kind of like your example if you provide the -ldebug
command line option when you run the test, e.g.,

 newt -ldebug test net/nimble/host
 Executing test:
/home/ccollins/repos/mynewt/core/bin/targets/unittest/net_nimble_host_test/test/net/nimble/host/test/net_nimble_host_test
 2016/10/03 08:00:49 [DEBUG]
/home/ccollins/repos/mynewt/core/bin/targets/unittest/net_nimble_host_test/test/net/nimble/host/test/net_nimble_host_test
 2016/10/03 08:00:50 [DEBUG] o=[pass]
 ble_att_clt_suite/ble_att_clt_test_tx_find_info
 [pass] ble_att_clt_suite/ble_att_clt_test_rx_find_info
 [pass] ble_att_clt_suite/ble_att_clt_test_tx_read
 [...]
 [pass] 
ble_sm_sc_test_suite/ble_sm_sc_peer_nc_iio1_rio1_b1_iat2_rat2_ik3_rk3
 [pass] 
ble_sm_sc_test_suite/ble_sm_sc_peer_pk_iio2_rio0_b1_iat2_rat2_ik7_rk3
 [pass] 
ble_sm_sc_test_suite/ble_sm_sc_us_jw_iio3_rio3_b1_iat2_rat2_ik3_rk3
 [pass] 
ble_sm_sc_test_suite/ble_sm_sc_us_nc_iio1_rio1_b1_iat2_rat2_ik3_rk3
 [pass] 
ble_sm_sc_test_suite/ble_sm_sc_us_pk_iio2_rio0_b1_iat2_rat2_ik7

Re: Version on statistics structures

[Kevin Townsend]

Generating a map file during 'create-image' makes a lot of sense.  I 
haven't tried without names enabled, but  a .json file would make life 
easier compared to digging around in an .elf file or something similar.



On 04/10/16 20:06, Sterling Hughes wrote:
As I’m going through the statistics module and documenting it, one 
thing that occurs to me is for the case where STATS_NAME_ENABLE = 0, 
and statistic names are omitted, we should create the statistic name 
-> number mapping, in the target’s directory, in a machine parseable 
format (i.e. JSON.) I’m thinking we should put this into the 
manifest.json when we create an image, that way if a management system 
wants to ingest this data, it can easily take this information, along 
with the created image.


I thought about versioning the structure, however decided against it, 
because:


- Manually updating the statistics version when it changes is error 
prone, because people forget to change version numbers


- I think it is likely that any management system or user who wants to 
interpret these values, will likely have the image definition for that 
image, _AND_ will probably not do an exhaustive search of all images, 
to find one that has a matching definition of version of statistics 
that are found on that device.  (likely:)


Comments welcome.  Below is the documentation I’ve written at the top 
of the module.


NOTE: Newt does not yet support generating statistics mappings in the 
manifest.json, I’m adding that support to create-image now.

Re: Version on statistics structures

[Kevin Townsend]

On 04/10/16 21:01, Sterling Hughes wrote:
Also, unless people object I’m going to make the stats_name_map a 
const structure (so it can be located in .text), pack it, as the name 
map is rarely referenced, and it should save a bunch of space given 
the id is only 16-bits.

+1

Re: newtmgr stat command error

[Kevin Townsend]

Perhaps I need to dig into golang :) ... though there isn't a lot of 
room left up there for new languages anymore and I think C and Python 
have pretty much driven everything else out over the years


K.


On 26/09/16 15:12, David G. Simmons wrote:

I reported this bug months ago. Seems the newtmgr does not check for required 
arguments/length. The stat command requires arguments, but newtmgr does not 
check that arguments are present, and then 'index out of range' exception.

dg


On Sep 25, 2016, at 5:11 PM, Kevin Townsend <ke...@adafruit.com> wrote:

Should the `stat` command in newtmgr be behaving similarly to the console 
command where it returns a list of all stat entries available?

That was the behaviour I was expecting, though I also tried `stat list`, `stat 
show` and `stat all` or just `stat` before looking at the documentation (which 
didn't elaborate, but not biggie there are more important problems to solve!).

It seems there might be an error here:

   $ newtmgr -c serial1 stat
   panic: runtime error: index out of range

   goroutine 1 [running]:
   panic(0x42965a0, 0xc820010080)
   /usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6
   mynewt.apache.org/newt/newtmgr/cli.statsRunCmd(0xc8200df800,
   0xc82000e820, 0x0, 0x2)
   /Users/ktown/prog/go/src/mynewt.apache.org/newt/newtmgr/cli/stats.go:57
   +0x946
   github.com/spf13/cobra.(*Command).execute(0xc8200df800,
   0xc82000e620, 0x2, 0x2, 0x0, 0x0)
   /Users/ktown/prog/go/src/github.com/spf13/cobra/command.go:565 +0x85a
   github.com/spf13/cobra.(*Command).ExecuteC(0xc8200de000,
   0xc8200df800, 0x0, 0x0)
   /Users/ktown/prog/go/src/github.com/spf13/cobra/command.go:651 +0x55c
   github.com/spf13/cobra.(*Command).Execute(0xc8200de000, 0x0, 0x0)
   /Users/ktown/prog/go/src/github.com/spf13/cobra/command.go:610 +0x2d
   main.main()
   /Users/ktown/prog/go/src/mynewt.apache.org/newt/newtmgr/newtmgr.go:25
   +0x25

When I run with a valid name things are fine:

   $ newtmgr -c serial1 stat ble_svc_dis
   Return Code = 0
   Stats Name: ble_svc_dis
  manufacturer_reads_stat: 0
  model_reads_stat: 0
  serial_reads_stat: 0
  firmware_rev_reads_stat: 0
  hardware_rev_reads_stat: 0
  software_rev_reads_stat: 0

It seems there isn't currently a way to know via newtmgr how many stat entries 
are present, although this information is available in the console version.

I'm running the latest newtmgr available in the master branch of the repo.

K.

PS: The stats module is great, BTW. The reason I enjoy using Mynewt so much is 
the (sadly uncommon) focus on real product deployments. Details like this make 
life so much easier when devices are out in the wild, but those 'details' tend 
to be forgotten in a lot of tools and platforms or a sad afterthought. So many 
RTOSes and platforms are overly academic and it's refreshing that the design 
decisions with Mynewt are being informed by the realities of real world product 
releases, and having to debug things in the field.


--
David G. Simmons
(919) 534-5099
Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin 
<http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub 
<http://github.com/davidgs>
/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
  * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
  * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!

There are only 2 hard things in computer science: Cache invalidation, naming 
things, and off-by-one errors.

newtmgr stat command error

[Kevin Townsend]

Should the `stat` command in newtmgr be behaving similarly to the 
console command where it returns a list of all stat entries available?


That was the behaviour I was expecting, though I also tried `stat list`, 
`stat show` and `stat all` or just `stat` before looking at the 
documentation (which didn't elaborate, but not biggie there are more 
important problems to solve!).


It seems there might be an error here:

   $ newtmgr -c serial1 stat
   panic: runtime error: index out of range

   goroutine 1 [running]:
   panic(0x42965a0, 0xc820010080)
   /usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6
   mynewt.apache.org/newt/newtmgr/cli.statsRunCmd(0xc8200df800,
   0xc82000e820, 0x0, 0x2)
   /Users/ktown/prog/go/src/mynewt.apache.org/newt/newtmgr/cli/stats.go:57
   +0x946
   github.com/spf13/cobra.(*Command).execute(0xc8200df800,
   0xc82000e620, 0x2, 0x2, 0x0, 0x0)
   /Users/ktown/prog/go/src/github.com/spf13/cobra/command.go:565 +0x85a
   github.com/spf13/cobra.(*Command).ExecuteC(0xc8200de000,
   0xc8200df800, 0x0, 0x0)
   /Users/ktown/prog/go/src/github.com/spf13/cobra/command.go:651 +0x55c
   github.com/spf13/cobra.(*Command).Execute(0xc8200de000, 0x0, 0x0)
   /Users/ktown/prog/go/src/github.com/spf13/cobra/command.go:610 +0x2d
   main.main()
   /Users/ktown/prog/go/src/mynewt.apache.org/newt/newtmgr/newtmgr.go:25
   +0x25

When I run with a valid name things are fine:

   $ newtmgr -c serial1 stat ble_svc_dis
   Return Code = 0
   Stats Name: ble_svc_dis
  manufacturer_reads_stat: 0
  model_reads_stat: 0
  serial_reads_stat: 0
  firmware_rev_reads_stat: 0
  hardware_rev_reads_stat: 0
  software_rev_reads_stat: 0

It seems there isn't currently a way to know via newtmgr how many stat 
entries are present, although this information is available in the 
console version.


I'm running the latest newtmgr available in the master branch of the repo.

K.

PS: The stats module is great, BTW. The reason I enjoy using Mynewt so 
much is the (sadly uncommon) focus on real product deployments. Details 
like this make life so much easier when devices are out in the wild, but 
those 'details' tend to be forgotten in a lot of tools and platforms or 
a sad afterthought. So many RTOSes and platforms are overly academic and 
it's refreshing that the design decisions with Mynewt are being informed 
by the realities of real world product releases, and having to debug 
things in the field.

Re: newtmgr over Serial

[Kevin Townsend]

Hi Marko,

Thanks for the reply ... I wasn't sure what to do with the console/stub 
dependency though ... where should this be set?


#
# Define BOOT_SERIAL in target features to include serial downloader.
# And uncomment 'libs/console/stub' from pkg.deps.
#

This builds properly but when I flash the bootloader on an nRF52 with an 
app, the app doesn't start up, but disabling BOOT_SERIAL it runs fine, 
so I'm clearly defining the lib/console/stub dependency in the wrong 
place (clear because I'm getting conflict alerts to the console libs 
when I compile).


I had this in the target.yml file for the bootloader target, but that 
doesn't seem to be correct:


target.deps: "@apache-mynewt-core/libs/console/stub"

K.

On 23/09/16 20:56, marko kiiskila wrote:

That all looks good.
boot_serial package is supposed to be small.

—8<—cut-start—   
[marko@IsMyLaptop:~/src/incubator-mynewt-blinky]$ newt target show boot_mkr1000
targets/boot_mkr1000
 app=@apache-mynewt-core/apps/boot
 bsp=@mynewt-arduino-zero/hw/bsp/arduino_mkr1000
 build_profile=optimized
 features=BOOT_SERIAL
—8<—cut end —

Indeed, you need to add  3 defines to your BSP. For testing, I only added
those to Arduino MKR1000 BSP. As you’ve probably figured out, these are for
the pin to use, and whether you want that pin pulled up/down, and whether
to compare pin value against 0 or 1.

—8< — cut start —
[marko@IsMyLaptop:~/src/incubator-mynewt-blinky]$ grep BOOT_SERIAL 
repos/mynewt-arduino-zero/hw/bsp/arduino_mkr1000/include/bsp/bsp.h
#ifdef BOOT_SERIAL
#define BOOT_SERIAL_DETECT_PIN  43
#define BOOT_SERIAL_DETECT_PIN_CFG  GPIO_PULL_UP
#define BOOT_SERIAL_DETECT_PIN_VAL  0
—8< — cut end —
I thought I sent an email about this, but maybe it was missing these details. 
Sorry
about that.

Now that we can take interrupts even without OS, the size of the bootloader with
serial support could be made smaller. At the moment it still starts the OS, 
while the
boot_serial stuff could just spin in a loop waiting for input.
That would be attractive here, because then you could have the bootloader 
smaller
than 16k (looks like it’s really close for your BSP).


On Sep 23, 2016, at 11:22 AM, Kevin Townsend <ke...@adafruit.com> wrote:

Hi Sterling,

I saw the note on the dependency, but is the target the right place to be 
adding the dep entry as follows:

   $ newt target set bootloader
   deps="@apache-mynewt-core/libs/console/stub"

Adding the two missing defines at the BSP level for BOOT_SERIAL gets this 
building at least

- BOOT_SERIAL_DETECT_PIN
- BOOT_SERIAL_DETECT_PIN_CFG

The size still seems reasonable to me (<32KB), though I haven't tested this yet 
to see if it's being built correctly and the warning is setting an alarm bell off 
for me, but I'll test shortly:

   $ newt size bootloader
   Warning: API conflict: console (libs/console/stub <-> libs/console/full)
   Warning: API conflict: console (libs/console/stub <-> libs/console/full)
  FLASH RAM
 23 222 *fill*
   2136  32 baselibc.a
3302128 boot.a
   1141  12 boot_serial.a
   17171132 bootutil.a
 64   0 cmsis-core.a
 20   1 config.a
124   0 crt0.o
  8   0 crti.o
 16   0 crtn.o
640 512 feather52.a
983 196 full.a
634   8 hal.a
 80   0 libg.a
   1436   0 libgcc.a
   1200   0 mbedtls.a
   1837  40 nrf52xxx.a
   3322 809 os.a
945   0 util.a

   objsize
   text   databssdechex filename
  16644128   4532  21304   5338  
bin/bootloader/apps/boot/boot.elf

K.


On 23/09/16 17:57, Sterling Hughes wrote:

Hi Kevin,

I think (and I’ll let Marko chime in here), you can use the boot_serial package 
to achieve this (apache-mynewt-core/libs/boot_serial.)

It speaks the newtmgr protocol, but doesn’t require the shell task or an image 
to be programmed.  I think it will slightly explode the size of your boot 
loader, but that shouldn’t be a huge issue on the NRF52.

apps/boot/ has the following options.

#
# Define BOOT_SERIAL in target features to include serial downloader.
# And uncomment 'libs/console/stub' from pkg.deps.
#
pkg.deps.BOOT_SERIAL.OVERWRITE:
- libs/console/full
- libs/boot_serial
pkg.cflags.BOOT_SERIAL:
- -DBOOT_SERIAL

It’s (unfortunately) been awhile since I’ve tested this, but we’ll take a look 
today and make sure it’s still functioning (it should be.)

Sterling

On 23 Sep 2016, at 2:46, Kevin Townsend wrote:


Hi Will (and company),

Sorry to recycle an old thread, but I was just doing some testing with the 
bootloader on the latest release, and wanted to come back to the issue of 
having a purely serial option for flashing images in the bootloader. From my 
perspective there are a number of valid use cases around uploading an image

Re: newtmgr over Serial

[Kevin Townsend]

Hi Sterling,

I saw the note on the dependency, but is the target the right place to 
be adding the dep entry as follows:


   $ newt target set bootloader
   deps="@apache-mynewt-core/libs/console/stub"

Adding the two missing defines at the BSP level for BOOT_SERIAL gets 
this building at least


- BOOT_SERIAL_DETECT_PIN
- BOOT_SERIAL_DETECT_PIN_CFG

The size still seems reasonable to me (<32KB), though I haven't tested 
this yet to see if it's being built correctly and the warning is setting 
an alarm bell off for me, but I'll test shortly:


   $ newt size bootloader
   Warning: API conflict: console (libs/console/stub <-> libs/console/full)
   Warning: API conflict: console (libs/console/stub <-> libs/console/full)
  FLASH RAM
 23 222 *fill*
   2136  32 baselibc.a
3302128 boot.a
   1141  12 boot_serial.a
   17171132 bootutil.a
 64   0 cmsis-core.a
 20   1 config.a
124   0 crt0.o
  8   0 crti.o
 16   0 crtn.o
640 512 feather52.a
983 196 full.a
634   8 hal.a
 80   0 libg.a
   1436   0 libgcc.a
   1200   0 mbedtls.a
   1837  40 nrf52xxx.a
   3322 809 os.a
945   0 util.a

   objsize
   text   databssdechex filename
  16644128   4532  21304   5338   
   bin/bootloader/apps/boot/boot.elf


K.


On 23/09/16 17:57, Sterling Hughes wrote:

Hi Kevin,

I think (and I’ll let Marko chime in here), you can use the 
boot_serial package to achieve this 
(apache-mynewt-core/libs/boot_serial.)


It speaks the newtmgr protocol, but doesn’t require the shell task or 
an image to be programmed.  I think it will slightly explode the size 
of your boot loader, but that shouldn’t be a huge issue on the NRF52.


apps/boot/ has the following options.

#
# Define BOOT_SERIAL in target features to include serial downloader.
# And uncomment 'libs/console/stub' from pkg.deps.
#
pkg.deps.BOOT_SERIAL.OVERWRITE:
- libs/console/full
- libs/boot_serial
pkg.cflags.BOOT_SERIAL:
- -DBOOT_SERIAL

It’s (unfortunately) been awhile since I’ve tested this, but we’ll 
take a look today and make sure it’s still functioning (it should be.)


Sterling

On 23 Sep 2016, at 2:46, Kevin Townsend wrote:


Hi Will (and company),

Sorry to recycle an old thread, but I was just doing some testing 
with the bootloader on the latest release, and wanted to come back to 
the issue of having a purely serial option for flashing images in the 
bootloader. From my perspective there are a number of valid use cases 
around uploading an image on an empty device (other than the 
bootloader) over UART or USB CDC, but others may disagree.


This would provide an inexpensive mechanism to debrick boards, for 
example, as well as a useful tool for production environments where 
you don´t have the financial or practical means to send a half dozen 
commercially licensed JLink to your assembly house or somewhere in 
China for testing then flashing.


Being able to run something like this with ONLY the bootloader 
present would be a big plus I think:


$ newtmgr -c serial image upload 
bin/bleuart/apps/bleuart/bleuart.elf.bin


As things stand today, you can only do this (I think, please correct 
me if I´m wrong) if you already have a valid image flashed and shell 
support enabled for newtmgr.


Is there an obstacle anyone can see about why this wouldn't be 
practical to implement with only the bootloader present? We've been 
focused on application level code and the peripheral side of nimble 
so I haven't looked at the bootloader code at all, but will have a 
look to try to get a better sense of the requirements here to use it 
with serial without any sort of shell support on the application side.


K.

On 08/06/16 23:59, will sanfilippo wrote:

+1

Guess that is my one cent opinion :-) Wouldnt be hard to do and is 
definitely a handy option for a certain group of folks. BTW, and 
this is a minor detail, I am not so much for polling a pin; the 
bootloader can look at the serial port for a certain sequence of 
characters. If it sees them it enters download mode. If it doesnt 
see anything it likes after that (or doesnt see that sequence), it 
tries to boot an image. If it cant, it just cycles back. If it boots 
a valid image, all good. If it boots a bricked image, you just gotta 
power cycle it. Shouldnt increase boot time too much (which is 
something to keep in mind imo).



On Jun 8, 2016, at 12:42 PM, marko kiiskila <ma...@runtime.io> wrote:

I’m convinced that we should have an option for using standalone 
boot loader

with which you can upload images. These are valid use cases.

We should make that happen.

Re: newtmgr over Serial

[Kevin Townsend]

Hi Sterling,

Thanks for the heads up. I added the cflag via '$ newt target set 
bootloader cflags=-DBOOT_SERIAL' and I can see that it exists in 
'/repos/apache-mynewt-core/libs/boot_serial', though when I try to build 
I'm getting:


   $ newt build bootloader
   Building target targets/bootloader
   Compiling boot.c
   Error: boot.c:33:37: fatal error: boot_serial/boot_serial.h: No such
   file or directory
 #include 
 ^
   compilation terminated.

The target definition seems fine though (I'm on the 'master' branch just 
as a FYI):


   $ newt target show
   targets/bootloader
app=@apache-mynewt-core/apps/boot
bsp=hw/bsp/feather52
build_profile=optimized
cflags=-DBOOT_SERIAL

I also tried 'newt install' and 'newt upgrade' to refresh the local 
setup but same results. Perhaps I need to update something in the local 
BSP, though ... I'll have a look at that.


Thanks for the heads up about this library, though. Somehow it never 
ended up on my radar, but I'll try to get this building and test it out 
this weekend.


BR,

Kevin

Re: Is anyone out there still using a nrf52 Preview (stress on the word Preview!) Dev kit?

[Kevin Townsend]

My vote would be to  remove it to push people to production parts and kits.

Le vendredi 23 septembre 2016, will sanfilippo  a écrit :

> Hello:
>
> The first dev kits produced by nordic which used the nrf52 were called
> Preview Development Kits. They have a PCA of 10036 on them (the large white
> label on the debugger chip contains the PCA). Just as an FYI:
>
> PCA 10028: nrf51DK
> PCA 10036: nrf52PDK *** Preview Dev Kit ***
> PCA 10040: nrf52DK
>
> The reason I am asking is that I think support for this development kit
> should be dropped. That version of the chip had many issues and keeping
> support for it seems to be a waste of time.
>
> Do folks have any objections?

Re: Switching to Master

[Kevin Townsend]

No, you aren't missing anything - we should have added a new pointer for
master.  Thanks for catching this!

For reference, here is the current apache-mynewt-core repository.yml
file:

 repo.name: apache-mynewt-core
 repo.versions:
 "0.0.0": "develop"
 "0.7.9": "mynewt_0_8_0_b2_tag"
 "0.8.0": "mynewt_0_8_0_tag"
 "0.9.0": "mynewt_0_9_0_tag"
 "0-latest": "0.9.0"
 "0-dev": "0.0.0"
 "0.8-latest": "0.8.0"
 "0.9-latest": "0.9.0"

I can't think of a new pointer name for master.  Since this is just a
temporary change, I'm thinking we should change 0-dev to point to
master.  0.0.0 would still point to develop.

0-dev seems to match the intent based on the comments here, and seems 
like a good match to me.


I did try pointing to "master" with a new entry but got the error below, 
but perhaps it is looking for a tag not a branch name:


   $ newt upgrade
   Error: Unknown stability (master) in version 0-master


I simply added '"0-master": "master"' as a quick test before deleting 
the project.state file and running the upgrade command above.


K.

Re: Low power design

[Kevin Townsend]

Hi Sterling,

I'm sure it's in the planning, but there should definately be some sort 
of callback at the BSP level before going into a lower sleep state (deep 
sleep or halt), as well as after wakeup.  This will allow you to 
configure pins accordingly to disable external pullups, avoid as much 
leakage current on the GPIOs as possible, perhaps switch to a lower 
voltage if you're using a special dual voltage regulator to save power, 
etc. And then when we wakeup the wake callback can be used to put the 
pins back to their pre-sleep state.


I assume that's what you meant though by "application hook for each 
system state", but we should probably be aware of both entering and 
exiting a specific state since there is BSP specific work on both ends.


K.


On 08/09/16 19:13, Sterling Hughes wrote:

Howdy,

I’m finally getting to implementing the low power support in mynewt, 
and I was hoping for comments from folks on the list before I get 
started.  I chatted with will, marko and paul about this a bit, and we 
think this is a reasonable set of first features, although we all 
wanted to mull it over.


We came down to the following set of goals for the OS:

- Provide standard definitions for drivers to provide the following 
functionality:

- On
- Off
- Suspend
- Resume


Where On turns on the peripheral and initializes it, Off turns off the 
peripheral such that it must be re-initialized, Suspend turns off the 
peripheral, but without requiring it to be fully re-initialized, and 
Resume takes the peripheral out of the suspend state.


- Provide a standard set of system states that users of Mynewt can 
transition the kernel into, in order to provide low power operation.  
This includes:

- Running
- Sleep
- Deep Sleep
- Halt

Where Running is normal operation, Sleep is WFI (processor off, 
interrupts enabled), Deep Sleep represents a state where all drivers 
are suspended, and Halt is the processor is essentially turned off.


- Allow user defined applications to set power policy based upon the 
system states: i.e. an application hook for each system state where 
additional behavior can be defined.


- (future) Provide a method to newt by which certain tasks memory can 
be linked earlier in RAM, so that for systems where RAM suspension is 
desired, a core set of system services can be located there, while 
other services can be re-initialized in a second phase.
- This may cause some modification to the task structure to 
provide these “re-init” functions.


- Misc:
- We already provide a tickless kernel, so today the system sets a 
timer for the next kernel event and issues WFI in the idle task.
- We need to add the concept of power domains to the driver 
interface.  The goal here is to manage releasing system power domains 
when all components that are using a specific power domain, release 
that domain.


Some questions:

- General questions/comments are welcome.

- What are the common use cases people have seen in the past?

- Do these designs address those use cases?

- What parts of the low power designs that you’ve done in the past 
have been particularly tricky?  What were the most important 
considerations that you made in those cases?


Best,
Sterling

Re: HAL & I2C redux

[Kevin Townsend]

I'd have to look at all the datasheets for the support MCUs, but some 
MCUs definitely don't play well with clock stretching, I just wish I 
could remember some examples quickly.  We've run into problems with 
popular SoCs or ICs like the BNO055, which use CS.


See this for example for the ESP (though this is just from a quick 
search on Google, I haven't tried): 
http://www.esp8266.com/viewtopic.php?p=41401


K.


On 25/08/16 20:14, marko kiiskila wrote:

I was wondering about clock stretching as well, and if it all targets
do it properly. If not, maybe there should be a way of controlling
the data clocking speed. At the moment I read the API such that the
MCU would try to clock out data as fast as it can?


On Aug 25, 2016, at 10:45 AM, Kevin Townsend <ke...@adafruit.com> wrote:

Hi Sterling (et al),

start() and stop() and probably clearer to anyone familiar with I2C, and it 
removes any ambiguity about if cached data is also being sent. Or the single 
control() command with a parameter seems fine as well, though that's just my 
opinion.

Something concerning blocking or non blocking calls as well ... I think having 
a user-defined timeout makes sense (as Marko mentioned) with an appropriate 
return code on timeout, but the API should also keep clock stretching in mind 
(http://www.i2c-bus.org/clock-stretching/). I'm not sure what the solution 
would be, though. If the I2C slave uses clock stretching and is holding the 
clock low, that's still in a valid state for the transaction, but what takes 
priority if the timeout expires first? I suppose the explicit timeout should 
take precedence and the transaction should be aborted, but it's worth 
considering. We've also had issues with clock stretching on some HW since not 
every I2C peripheral on every MCU handles it properly either ... it might be 
worth having a flag and the MCU abstraction level to indicate of the I2C 
peripheral supports clock stretching, just to know when working with drivers 
that require it.

K.


On 25/08/16 19:36, Sterling Hughes wrote:

Hi Kevin,

On 25 Aug 2016, at 9:49, Kevin Townsend wrote:


Although, this will also depend on how things are implemented in the .c code 
... I only see the header at present. But from experience some sensors require 
the stop to be handled differently between multiple read or writes, so it's 
worth considering how to keep the STOP condition flexible and under user 
control since there isn't a one size fits all approach for every I2C sensor out 
there.



Yeah,  most of the vendor APIs I see have the stop bit as an option (or by 
default) after a write() command, whereas our HAL APIs use begin() and end() to 
generate stop conditions.  Frankly, I think it might make more sense to label 
these start() and stop(), because a lot of the I2C protocols can have multiple 
start conditions and multiple stop conditions.  Or maybe a control() command 
that takes either START or STOP, to make it clear that it’s not modifying 
transaction state in anyway.

Sterling

Re: HAL & I2C redux

[Kevin Townsend]

Hi Sterling (et al),

start() and stop() and probably clearer to anyone familiar with I2C, and 
it removes any ambiguity about if cached data is also being sent. Or the 
single control() command with a parameter seems fine as well, though 
that's just my opinion.


Something concerning blocking or non blocking calls as well ... I think 
having a user-defined timeout makes sense (as Marko mentioned) with an 
appropriate return code on timeout, but the API should also keep clock 
stretching in mind (http://www.i2c-bus.org/clock-stretching/). I'm not 
sure what the solution would be, though. If the I2C slave uses clock 
stretching and is holding the clock low, that's still in a valid state 
for the transaction, but what takes priority if the timeout expires 
first? I suppose the explicit timeout should take precedence and the 
transaction should be aborted, but it's worth considering. We've also 
had issues with clock stretching on some HW since not every I2C 
peripheral on every MCU handles it properly either ... it might be worth 
having a flag and the MCU abstraction level to indicate of the I2C 
peripheral supports clock stretching, just to know when working with 
drivers that require it.


K.


On 25/08/16 19:36, Sterling Hughes wrote:

Hi Kevin,

On 25 Aug 2016, at 9:49, Kevin Townsend wrote:

Although, this will also depend on how things are implemented in the 
.c code ... I only see the header at present. But from experience 
some sensors require the stop to be handled differently between 
multiple read or writes, so it's worth considering how to keep the 
STOP condition flexible and under user control since there isn't a 
one size fits all approach for every I2C sensor out there.





Yeah,  most of the vendor APIs I see have the stop bit as an option 
(or by default) after a write() command, whereas our HAL APIs use 
begin() and end() to generate stop conditions.  Frankly, I think it 
might make more sense to label these start() and stop(), because a lot 
of the I2C protocols can have multiple start conditions and multiple 
stop conditions.  Or maybe a control() command that takes either START 
or STOP, to make it clear that it’s not modifying transaction state in 
anyway.


Sterling

Re: HAL & I2C redux

[Kevin Townsend]

Although, this will also depend on how things are implemented in the .c 
code ... I only see the header at present. But from experience some 
sensors require the stop to be handled differently between multiple read 
or writes, so it's worth considering how to keep the STOP condition 
flexible and under user control since there isn't a one size fits all 
approach for every I2C sensor out there.



On 25/08/16 18:46, Kevin Townsend wrote:
You might want to consider how to handle the STOP condition ... some 
sensors have different behaviour about when the stop condition should 
or shouldn't be set.  See the Arduino Wire library for example, who 
make the stop condition optional when you end the transaction: 
https://www.arduino.cc/en/Reference/WireEndTransmission


Kevin


On 25/08/16 01:37, Sterling Hughes wrote:

Hi,

While we’re doing the HAL changes, I’d like folks to check out the 
I2C APIs, and provide any comments they have:


https://github.com/apache/incubator-mynewt-core/blob/sterly_refactor/hw/hal/include/hal/hal_i2c.h 



I was relatively happy with these APIs, except for maybe the 
hal_i2c_master_data: I think write/read should just take these three 
as arguments: but I didn’t see this as significant enough to change.


However, the one difference between SPI (new SPI HAL APIs) and UART 
is that I2C does not have a non-blocking mode.  I didn’t think this 
was a huge issue, because I assume that I2C is likely going to be 
mostly slow communication in a low-priority task context.  However, 
I’m interested to know if folks think an interrupt based/non-blocking 
API is desirable for I2C, and worth the implementation overhead for 
people developing the HAL.


Also, folks should feel free to review the new HAL which is here:

https://github.com/apache/incubator-mynewt-core/tree/sterly_refactor/hw/hal/include/hal 



With the caveat that Will’s new SPI HAL interface isn’t committed, 
and we are missing a watchdog HAL (coming soon.)




Sterling

Re: HAL & I2C redux

[Kevin Townsend]

You might want to consider how to handle the STOP condition ... some 
sensors have different behaviour about when the stop condition should or 
shouldn't be set.  See the Arduino Wire library for example, who make 
the stop condition optional when you end the transaction: 
https://www.arduino.cc/en/Reference/WireEndTransmission


Kevin


On 25/08/16 01:37, Sterling Hughes wrote:

Hi,

While we’re doing the HAL changes, I’d like folks to check out the I2C 
APIs, and provide any comments they have:


https://github.com/apache/incubator-mynewt-core/blob/sterly_refactor/hw/hal/include/hal/hal_i2c.h 



I was relatively happy with these APIs, except for maybe the 
hal_i2c_master_data: I think write/read should just take these three 
as arguments: but I didn’t see this as significant enough to change.


However, the one difference between SPI (new SPI HAL APIs) and UART is 
that I2C does not have a non-blocking mode.  I didn’t think this was a 
huge issue, because I assume that I2C is likely going to be mostly 
slow communication in a low-priority task context.  However, I’m 
interested to know if folks think an interrupt based/non-blocking API 
is desirable for I2C, and worth the implementation overhead for people 
developing the HAL.


Also, folks should feel free to review the new HAL which is here:

https://github.com/apache/incubator-mynewt-core/tree/sterly_refactor/hw/hal/include/hal 



With the caveat that Will’s new SPI HAL interface isn’t committed, and 
we are missing a watchdog HAL (coming soon.)




Sterling

Re: HAL SPI

[Kevin Townsend]

Regarding slaves: I thought that I mentioned which interfaces would be used by 
a slave? The slave interfaces are fairly basic, admittedly, and there might be 
other interfaces that one might want to have, but they should be usable by a 
slave. I will give my mail another read through to see if I left something out 
but I certainly intended to provide (at least basic) slave functionality.

Oh, ugg, my bad I see it now. :)

I don't often use slave, but every time I do need it for SPI or I2C it 
always feels like an after thought by the silicon or IP vendors. Glad to 
see that's included as well then.


K.

Re: HAL SPI

[Kevin Townsend]

Hi Will,

You've probably already included this in the spi_config struct, but just 
in case I wanted to make sure you can dynamically change the SPI speed 
even after the spi bus has been initialized once?


The use case I have in mind is SD cards communicating over the SPI bus 
where you need to start the communication at a fairly slow clock speed, 
and once everything is setup you can jump to a much faster clock. I'm 
sure it's already in there though.


The callbacks below seem sensible to me based on the drivers I've 
written in the past.


Being able to bit-bang SPI at a lower level with any GPIO pins would be 
a nice plus, but hardly a deal breaker and that is a layer lower anyway.


It might be worth doing a quick review of the requirements around SPI 
slave as well, just to see if you can accommodate both in one HAL 
update? Slave isn't commonly used, but for end users who are making 
commercial modules or components that fit into a larger system, such as 
say a wireless IMU widget, you'll probably want to enable SPI slave to 
communicate with another MCU. It isn't worth delaying the current HAL 
update for this, but maybe just something to think about quickly to see 
if you can carve room out for that in a future update without 
introducing a BC?


I don't see any fundamental issues that would prevent me from using this 
for most SPI devices I've worked with in the past, though. That's just 
my opinion, mind you, and I'm sure other users will comment in more 
detail if they spot something.


K.


On 23/08/16 23:23, will sanfilippo wrote:

Hello:

We are considering a new HAL for the SPI. Following is the set of API we are 
considering for this HAL. Comments are appreciated. NOTE: the term “value” is 
used instead of “byte” as SPI interfaces may use more than 8 bits per value 
transferred.

NOTE: the spi_config structure is not shown here. It will contain the receive 
callback and the transmit done callback along with other generic SPI 
configuration.

typedef int (*hal_spi_rx_val)(void *arg, uint8_t val);
This is the prototype for the receive callback. Called for each received spi 
value at interrupt context when using the hal_spi_start_rx() API.

typedef void (*hal_spi_tx_done)(void *arg);
Callback when buffer transferred using the hal_spi_txrx API.

int hal_spi_init(int spi_num, void *cfg)
Initializes the spi with processor specific configuration. This does not turn 
on the SPI nor does it enable it from a power perspective.

int hal_spi_config(int spi_num, struct spi_config *cfg);
Used to configure generic SPI information after initialization. This will 
contain the receive callback as well as generic information like # data bits, 
baudrate, mode, etc. Also will include the callbacks.

int hal_spi_enable(int spi_num);
Turns the SPI on from a power perspective

int hal_spi_disable(int spi_num);
Turns the SPI off from a power perspective

uint16_t hal_spi_tx_val(int spi_num, uint16_t val);
Blocking call to send a value on the SPI. Returns the value received from the 
SPI.
MASTER: Sends the value and returns the received value from the slave.
SLAVE: Sets the value to send to the master when the master transfers a value. 
This value will be sent until another call to hal_spi_tx_val() is made. The 
return code is ignored for a slave.

int hal_spi_txrx(int spi_num, void *txbuf, void *rxbuf, int len);
Non-blocking call to send a buffer and also store the received values. For both 
the master and slave, the transmit done callback is executed at interrupt 
context when the buffer is sent. If rxbuf is NULL, no received values are 
stored. There is no callback per value when using this API
MASTER: master sends all the values in the buffer and stores the values in the 
receive buffer if rxbuf is not NULL.
SLAVE: Slave preloads the data to be sent to the master (values stored in 
txbuf) and places received data from master in rxbuf (if not NULL). No callback 
per received value; transmit done callback when len values are transferred.

int hal_spi_start_rx(int spi_num);
Enables the SPI receiver. Only valid for slaves. Returns error if the receive 
is not able to be started. There will be a receive callback per value.

int hal_spi_abort(int spi_num);
This aborts the current transfer but keeps the spi enabled.

void hal_spi_set_rx_val_cb(int spi_num, hal_spi_rx_val rx_val_cb, void *arg);
Sets the callback executed when a slave receives a value. Allowed to be called 
when the spi is enabled but not when a transfer is in progress.

void hal_spi_set_tx_done_cb(int spi_num, hal_spi_tx_done tx_done_cb, void *arg);
Sets the callback executed the buffer is transferred by the master or slave 
using the hal_spi_txrx API. Allowed to be called when the SPI is enabled but 
not during a transfer.

Re: 3rd party SDKs, and Interrupt vectors on NRF52

[Kevin Townsend]

 Hi Will,

That's a good point. I've usually worked with very simple boot loaders in
the past. One of my biggest problems with the SD architecture (and
motivation for moving to mynewt and nimble) was the lack of strict timing
and direct interrupt control at the app level due to the interrupt handling
in the SD.

But one or two clear examples of how to register your interrupt handler
should probably be clearly documented with the current approach then since
it differs from what most customers are used to(?) with CMSIS and most
vendor supplied example code.

I don't have strong feelings either way myself though and the bootloader
argument is valid if you want to handle systick interrupts in both chunks
of code for example or radio and SPI events in an advanced bootloader.

K.

Le mercredi 27 juillet 2016, will sanfilippo <wi...@runtime.io> a écrit :

> So how does a bootloader that uses interrupts and an application that uses
> the same interrupt work? If you have interrupt vectors in .text, only one
> “application” can own that interrupt vector. Of course, the interrupt
> vector in . text can look up a function pointer to call, but in that case
> you might as well just register the interrupt vector.
>
>
> > On Jul 26, 2016, at 7:45 PM, Kevin Townsend <ke...@adafruit.com
> <javascript:;>> wrote:
> >
> >
> >> Bonsoir,
> > Bilingual, impressive! :P
> >> I’m OK with NVIC_SetVector(), and indeed, the driver that uses Nordic’s
> function can call that in the driver init function — however, I think it’s
> worth understanding why we want this located in RAM.  It seems reasonable
> to me to fix the interrupt vectors in .text and avoid the NVIC_SetVector()
> call — it seems like we’re spending a bunch of RAM on something that will
> never change dynamically.
> >>
> >> Thoughts?
> > Personally, I can't remember the last time I had the interrupt vectors
> anywhere except fixed in .text.  I suspect 99% of the people working with
> ARM Cortex devices are also familiar with and probably are sort of
> expecting the typical 'weak' approach pushed by most CMSIS based vendor
> startup code. That said ... I certainly don't feel strongly about this
> either way, but you probably are losing a small chunk of SRAM for something
> 0.1% of users might actually make meaningful use of.
> >
> > Kevin
>
>

Re: Mynewt BLEuart app

[Kevin Townsend]

Hi Vipul,

This is a helpful addition I think.  We actually implemented some helper 
code around this and we were going to send in a pull request, but you 
beat us to it.  We have a number of apps available to work with the 
Nordic UART service since it's the easiest way we've found to get data 
from a BLE peripheral to another device without having to know a lot 
about BLE or mobile/desktop development.


There are apps for OS X, iOS, and Android (all available in the app 
stores if you search for 'Bluefruit'), as well as other reference 
applications using Nordic's UART service here: 
https://learn.adafruit.com/introducing-adafruit-ble-bluetooth-low-energy-friend/software-resources


The OS X app is quite new, but I find it useful to get sensor data onto 
the laptop for plotting or parsing and you can then export to various 
file formats.


Best regards,
Kevin


On 22/07/16 04:13, Vipul Rahane wrote:

Hello all,

I have been working on a Mynewt BLEuart library and application which can work 
with Nordic’s nRF Uart app on iOS and Android.

I have committed an initial version of the same which you can find at 
http://bit.do/mynewtBLEuart  which is part of my fork of 
apache Mynewt core at https://github.com/vrahane/incubator-mynewt-core/tree/develop 


Currently it contains a library in libs/bleuart which takes strings coming via 
uart and transmits them over BLE and vice-versa.

Soon to come would be a binary version of this. This will help application 
protocol developers in their development phase as well as in production.

Regards,
Vipul Rahane

Re: NimBLE host - flat bufs vs. mbufs

[Kevin Townsend]

We were actually discussing some buffer problems internally this 
weekend, debating mbuf versus other solutions, etc.  We needed a simple 
FIFO to store data from Nordic's BLE UART service until the end user 
wants to access it, and we recycled some existing FIFO code we 
implemented for Nordic's SD. I was pushing to stick to standard buffers 
from Mynewt (to improves odds of having pull requests accepted), but 
dealing with storing 1 byte records means things like mbuf aren't very 
efficient.


I'm curious what you would have done for a simple FIFO where you are 
likely storing 1 or 2 bytes at a time?  If there isn't a good match, 
perhaps it's worth adding a very light weight FIFO or circular buffer 
that can deal with very small records efficiently, but I might be 
missing something that already exists?


Kevin

PS: Sorry, hijacking the discussion a bit and going off on a tangent, 
but I wanted to comment since buffers will be an important part of many 
libraries with Mynewt, or helper code around BLE services, and it would 
be good to make sure we have options for both convenience and 
performance/efficiency, just to avoid everyone reinventing the wheel.


On 19/07/16 15:38, will sanfilippo wrote:

I am +1 for mbufs. While they do take a bit of getting used to, I think 
converting the host to use them is the way to go, especially if they replace a 
large flat buffer pool.

I also think we should mention that mbufs use a fair amount of overhead. I dont 
think that applies here as the flat buffer required for an ATT buffer is quite 
large (if I recall). But if folks want to use mbufs for other things they 
should be aware of this.


On Jul 18, 2016, at 5:09 PM, Christopher Collins  wrote:

Hello all,

This email concerns the type of data buffers that the NimBLE host
requires the application to use.  There are several instances in which
buffers are passed between host and application.  For example, when the
application performs a GATT write procedure, it supplies the data to
be written by passing a buffer to the host.

The core of Mynewt presents two different mechanisms for implementing
data buffers:

1. Flat bufs
2. Mbufs

A flat buf is a pointer to a region of contiguous storage and is usually
accompanied by a length argument.  This is what the POSIX socket API
uses in its send() and recv() functions, for example.

Mbufs are dynamically resizable buffer chains.  They are described in
much more detail here:
http://mynewt.apache.org/latest/os/core_os/mbuf/mbuf/

The NimBLE host uses flat bufs, which makes it a bit of an outlier.
Among all the core network libraries, the host appears to be the only
one whose API is not based on mbufs.  In particular, I am comparing the
NimBLE host to mn_socket and newtmgr.

While it may be a bit painful to switch, I have convinced myself that it
ultimately makes more sense for the host to use mbufs instead of flat
bufs.  I put together two lists: reasons to prefer flat bufs and reasons
to prefer mbufs.

Pro flat buf:

* Simple interface.

Pro mbuf:

* Consistent with the rest of Mynewt core.

* More memory efficient (no need to pre-allocate a large ATT buffer; share
  memory with the rest of the system).

* More speed efficient (NimBLE host already uses mbufs internally;
  no need to convert between the two buffer types).

I ordered the items in the "pro mbuf" list from most to least important,
as I see it.  As much as I appreciate the simplicity of flat bufs, I
have to place a greater importance on consistency of the system as a
whole.

I have had a difficult time deciding on this issue, and I would be very
interested in hearing any and all thoughts on the subject.  Absent
overwhelming disapproval, I am going to get started on implementing this
change on a separate branch.  That way we will have something more
concrete to evaluate.

Thanks,
Chris

Re: Handling Notify/Indicate and CCCD Changes

[Kevin Townsend]

Hi Chris,

This looks great and I think adequately addresses the issues brought up 
earlier.


Sorry we haven't been able to provide some code in a pull request on our 
end yet, we're still coming to terms with the stack and codebase, but 
thanks for making these useful changes. We're hoping to submit some pull 
requests around specific BLE classes, and will be happy to contribute 
moving forward, but it will take some time to understand many of the 
nimble internals, etc.


Best regards,
Kevin

Re: Scan Response and Timing

[Kevin Townsend]

Hi Chris,


None of the example apps configure the scan response data (we should
change this).  It is relatively simple to configure response data: call
ble_gap_adv_rsp_set_fields()
(http://mynewt.apache.org/develop/network/ble/ble_hs/ble_gap/functions/ble_gap_adv_rsp_set_fields/).
This function is more or less identical to ble_gap_adv_set_fields(),
except it configures response data rather than advertising data.
That was easy then, thanks. Sorry for missing something obvious now that 
I see it. :(

PS: One other note with reference to the design guidelines from Apple:
many Apple devices are very specific about the timing requirements
(Section 3.5, 3.6, etc.) and devices that don't respect the limits may
not appear even if they are within the core specification limits. It
might be worth adding a section to the documentation on controlling the
key timing values (advertising interval, connection interval, etc.).
We're just looking into some of the details now that we have a basic app
running with DIS and Nordic UART, and I'm happy to contribute some
documentation in markdown as we track these details down, but it might
be worth considering having something easy to use in the core API to
control the various timing values in one central location.

Is there a particular API change that you are thinking of?  You're right
about there being are a lot of timing settings, and anything that makes
the API simpler would be an improvement.
We'll finish this quick demo app and I'll let you know if there are any 
specific suggestions, or we can push out a pull request if something 
makes a lot of sense and other people on the dev list can review it an 
comment.


I think having a really minimal nimble example with the following would 
help people, though, but perhaps it's something we can contribute since 
it's basically what we're working on locally:


- Match Apple's Bluetooth Low Energy timing restrictions (20ms 
connection interval, etc.)

- Local Name and TX Power in the Scan Response (as per Apple's suggestion)
- Device Information Service (Strong recommended by Apple, but generally 
just useful for tracking firmware revisions in mobile apps, etc.)
- Simple service example using notify or indicate (Nordic's BLE UART is 
a good and useful example that's easy to understand)


That's probably going to be the baseline requirement of any real world 
app in my opinion, and following Apple's recommendations works fine on 
Android devices as well (where you have a small minimum connection 
interval).


K.

Scan Response and Timing

[Kevin Townsend]

Are there any examples of how to setup a Scan Response while 
advertising? Apple, for example, recommends putting the Local Name and 
TX Power in the scan response to free up space.  See section 3.4 of 
https://developer.apple.com/hardwaredrivers/BluetoothDesignGuidelines.pdf


The big benefit here is that you can advertise the local name in the 
scan response, freeing up enough room in the main advertising packet for 
a 128-bit UUID if necessary, since primary services should always be 
advertised in the main payload. Most of the mobile apps from Nordic 
require that the Nordic BLE UART service's 128-bit UUID is advertised, 
for example, which doesn't leave a lot of space for other things.


It's no problem to dig into the code and figure this out, I just wanted 
to ping the list first to see if this was possible today before 
committing some resources to it.


Kevin

PS: One other note with reference to the design guidelines from Apple: 
many Apple devices are very specific about the timing requirements 
(Section 3.5, 3.6, etc.) and devices that don't respect the limits may 
not appear even if they are within the core specification limits. It 
might be worth adding a section to the documentation on controlling the 
key timing values (advertising interval, connection interval, etc.). 
We're just looking into some of the details now that we have a basic app 
running with DIS and Nordic UART, and I'm happy to contribute some 
documentation in markdown as we track these details down, but it might 
be worth considering having something easy to use in the core API to 
control the various timing values in one central location.