Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Tito Ciuro]

Hi Steven.

You're absolutely right.

Adding the sources to a real iOS project (standard Xcode 4, View-based iOS app) 
results in 885 KB. Great to see the linker/stripping process remove all this 
symbol info.

Thanks for the help,

-- Tito

On May 16, 2011, at 8:49 AM, Steven Parkes wrote:

> Well, for my part, I forgot that I'm not including the R-tree stuff at this 
> point. I do include FTS (that's why I have a custom build, not to mention the 
> latest WAL stuff.)
> 
> And to the extent it matters, I'm not using LLVM.
> 
> But I am building -O0 -ggdb. That doesn't do any inline or deadcode 
> elimination and it throws lots and lots and lots of symbol stuff in. Lots.  
> Really. Lots.
> 
> Unless I'm missing something, you're looking at something that doesn't 
> matter. The size of the .a is very weakly correlated to resulting size of the 
> executable. You don't ship the .a (or you certainly shouldn't be). It just 
> gets linked into your executable and then you strip your executable. Both the 
> linking and the stripping steps remove a ton of symbol information. Much of 
> the size of the sqlite .o's and .a's are symbols that are going to be dumped.
> 
> To give you an idea, as I said, my sqlite library is about 3M. My custom 
> openssl build is 17M. My own library is 37M. The final executable in 
> debugging mode is 7M before strip and 5M. Again, with -O0.
> 
> I'm pretty sure this is a non-issue.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Steven Parkes]

Well, for my part, I forgot that I'm not including the R-tree stuff at this 
point. I do include FTS (that's why I have a custom build, not to mention the 
latest WAL stuff.)

And to the extent it matters, I'm not using LLVM.

But I am building -O0 -ggdb. That doesn't do any inline or deadcode elimination 
and it throws lots and lots and lots of symbol stuff in. Lots.  Really. Lots.

Unless I'm missing something, you're looking at something that doesn't matter. 
The size of the .a is very weakly correlated to resulting size of the 
executable. You don't ship the .a (or you certainly shouldn't be). It just gets 
linked into your executable and then you strip your executable. Both the 
linking and the stripping steps remove a ton of symbol information. Much of the 
size of the sqlite .o's and .a's are symbols that are going to be dumped.

To give you an idea, as I said, my sqlite library is about 3M. My custom 
openssl build is 17M. My own library is 37M. The final executable in debugging 
mode is 7M before strip and 5M. Again, with -O0.

I'm pretty sure this is a non-issue.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Tito Ciuro]

Hi Richard,

On May 16, 2011, at 8:40 AM, Richard Hipp wrote:

> I took the amalgamation file (sqlite3.c) and compiled it thusly:
> 
>   gcc -Os -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -c sqlite3.c
> 
> The resulting binary size (as reported by the "size" command) is 392,203
> bytes.  That's 383 KiB.  I don't know how you are getting multi-megabyte
> builds

Darn. I wonder what settings in Xcode 4 are generating this size. I'll 
investigate more.

Thanks for the help,

-- Tito
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Richard Hipp]

On Mon, May 16, 2011 at 11:32 AM, Tito Ciuro  wrote:

> Hi Steven,
>
> OK. First of all, I messed up by compiling shell.c (which is included in
> the SQLite amalgamated distro.) Removing it brings the size to:
>
> GCC:
> - Debug: 3.1 MB
> - Release: 3.4 MB
>
> LLVM Compiler 2.0:
> - Debug: 3.7 MB
> - Release: 4 MB
>

I took the amalgamation file (sqlite3.c) and compiled it thusly:

   gcc -Os -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE -c sqlite3.c

The resulting binary size (as reported by the "size" command) is 392,203
bytes.  That's 383 KiB.  I don't know how you are getting multi-megabyte
builds


>
> This is what I did:
>
> 1) Create a new project in Xcode, iOS static library
> 2) Add the SQLite amalgamation sources (sqlite3.c, sqlite3.h, sqlite3ext.h)
> 3) Choose GCC
> 4) Build for Run (uses the Debug config)
> 5) Build for Archive (uses the Release config)
> 6) Choose LLVM
> 7) Repeat steps 4 and 5
>
> If you want to check the project, please contact me off-list and I'll
> gladly send it to you.
>
> Cheers,
>
> -- Tito
>
> On May 16, 2011, at 8:18 AM, Steven Parkes wrote:
>
> >> Compiling a static library of SQLite's amalgamated version weighs at
> about 4.3 MB
> >
> > Where are you coming up with this number? My .a is 2792KB and that's with
> both armv6 and armv7, debugging, and full symbols.
> >
> > I pull in sqlite3, openssl, about a billion other things, and plenty of
> my own code and my post-strip size of the executable is still on the order
> of 5MB and that's still with debugging turned on.
> >
> > I may be missing something, but I can't believe this a real issue.
> > ___
> > sqlite-users mailing list
> > sqlite-users@sqlite.org
> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Tito Ciuro]

Hi Simon,

No, I don't need it... but someone else might. I was considering adding FTS3 to 
my library in case someone needs to search across several text records. 
Depending on the table size, LIKE or GLOB would be very expensive because doing 
a full table scan would not be the most optimal approach, even if the database 
is not that large on an iPhone.

To be honest, it's more of a curiosity than actual need. I would not add it 
until it was needed, but I'd like to know what lies ahead.

Thanks Simon,

-- Tito

On May 16, 2011, at 8:26 AM, Simon Slavin wrote:

> 
> On 16 May 2011, at 4:06pm, Marco Bambini wrote:
> 
>> 4.3 MB seems really too big... you are probably building a debug version of 
>> the library.
> 
> Agreed.  I don't know what's wrong but nothing should take 4Meg.
> 
> Also, do you really need FTS3 ?  Try a simple search using LIKE or GLOB.  
> With the small amount of data your app will have (since it has to fit within 
> an iPhone anyway), is your search really too slow for normal use ?
> 
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Tito Ciuro]

Hi Steven,

OK. First of all, I messed up by compiling shell.c (which is included in the 
SQLite amalgamated distro.) Removing it brings the size to:

GCC:
- Debug: 3.1 MB
- Release: 3.4 MB

LLVM Compiler 2.0:
- Debug: 3.7 MB
- Release: 4 MB

This is what I did:

1) Create a new project in Xcode, iOS static library
2) Add the SQLite amalgamation sources (sqlite3.c, sqlite3.h, sqlite3ext.h)
3) Choose GCC
4) Build for Run (uses the Debug config)
5) Build for Archive (uses the Release config)
6) Choose LLVM
7) Repeat steps 4 and 5

If you want to check the project, please contact me off-list and I'll gladly 
send it to you.

Cheers,

-- Tito

On May 16, 2011, at 8:18 AM, Steven Parkes wrote:

>> Compiling a static library of SQLite's amalgamated version weighs at about 
>> 4.3 MB
> 
> Where are you coming up with this number? My .a is 2792KB and that's with 
> both armv6 and armv7, debugging, and full symbols.
> 
> I pull in sqlite3, openssl, about a billion other things, and plenty of my 
> own code and my post-strip size of the executable is still on the order of 
> 5MB and that's still with debugging turned on.
> 
> I may be missing something, but I can't believe this a real issue.
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Jean-Denis Muys]

On 16 mai 2011, at 17:01, Tito Ciuro wrote:

> Hello,
> 
> I have a question about SQLite running on iOS. If I'm not mistaken, SQLite on 
> iOS is not compiled with R*Tree and FTS3. Compiling a static library of 
> SQLite's amalgamated version weighs at about 4.3 MB, which represents almost 
> 25% of the 20 MB-per-app allowed on the App Store. For many, this is a major 
> setback because many apps can easily reach this limit.
> 
> My question is: since a "light" version of SQLite is already included in iOS, 
> would it be too complicated to build a static library with only R*Tree and 
> FTS3 support? The idea being of course that the app would link against iOS' 
> SQLite and the app's R*Tree/FTS3 library, thus reducing the app's footprint 
> considerably.
> 
> Are there dependencies that would make this attempt a nightmare? Has anyone 
> gone through this?

I have considered this, but not done it as it turned out that I could do 
without R-Tree after all for the time being.

But my initial investigation suggested that it would be possible to do so. I 
simply went through the amalgamation source code where it registers the R-Tree 
extension, and I realize that I could simply do the same thing, but using the 
R-Tree source file from the canonic source code.

I didn't check for FTS3, but I suppose it's handled similarly.

I haven't done it, so I can't be sure there is not hidden trap, but I can think 
of at least one pitfall: version compatibility.

you must register with the iOS SQLite a R-Tree extension that is compatible 
with - if not exactly the same as - the iOS version of SQLite. It's unclear to 
me whether the *current* version of R-Tree is compatible with the *current* iOS 
SQLite version.

you could think that the best idea is to go back to the same R-Tree version as 
iOS's. But even that might be problematic, as iOS is a moving target, and you 
risk your app breaking on a future iOS version... if it is not rejected by 
Apple in the first place for registering an extension to the system SQLite (I 
don't know that as I haven't tried).

I am not knowledgeable enough with SQLite to understand what is the best way to 
minimize compatibility risks between different versions of the core Library and 
its standard extension.

But saving 4.3 MB of app size doesn't seem valuable enough to me to run those 
risks. Even if those 4.3 MB make you break the 20 MB threshold.

Sorry for not being more helpful.

Jean-Denis

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Simon Slavin]

On 16 May 2011, at 4:06pm, Marco Bambini wrote:

> 4.3 MB seems really too big... you are probably building a debug version of 
> the library.

Agreed.  I don't know what's wrong but nothing should take 4Meg.

Also, do you really need FTS3 ?  Try a simple search using LIKE or GLOB.  With 
the small amount of data your app will have (since it has to fit within an 
iPhone anyway), is your search really too slow for normal use ?

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Tito Ciuro]

Hi Marco,

Oops. My bad. Building for Archive (Release version) has a size of 3.6 MB.

-- Tito

On May 16, 2011, at 8:06 AM, Marco Bambini wrote:

> 4.3 MB seems really too big... you are probably building a debug version of 
> the library.
> 
> --
> Marco Bambini
> http://www.sqlabs.com
> 
> 
> 
> 
> 
> 
> On May 16, 2011, at 5:01 PM, Tito Ciuro wrote:
> 
>> Hello,
>> 
>> I have a question about SQLite running on iOS. If I'm not mistaken, SQLite 
>> on iOS is not compiled with R*Tree and FTS3. Compiling a static library of 
>> SQLite's amalgamated version weighs at about 4.3 MB, which represents almost 
>> 25% of the 20 MB-per-app allowed on the App Store. For many, this is a major 
>> setback because many apps can easily reach this limit.
>> 
>> My question is: since a "light" version of SQLite is already included in 
>> iOS, would it be too complicated to build a static library with only R*Tree 
>> and FTS3 support? The idea being of course that the app would link against 
>> iOS' SQLite and the app's R*Tree/FTS3 library, thus reducing the app's 
>> footprint considerably.
>> 
>> Are there dependencies that would make this attempt a nightmare? Has anyone 
>> gone through this?
>> 
>> Thanks in advance,
>> 
>> -- Tito
>> ___
>> sqlite-users mailing list
>> sqlite-users@sqlite.org
>> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Steven Parkes]

> Compiling a static library of SQLite's amalgamated version weighs at about 
> 4.3 MB

Where are you coming up with this number? My .a is 2792KB and that's with both 
armv6 and armv7, debugging, and full symbols.

I pull in sqlite3, openssl, about a billion other things, and plenty of my own 
code and my post-strip size of the executable is still on the order of 5MB and 
that's still with debugging turned on.

I may be missing something, but I can't believe this a real issue.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Using SQLite with R*Tree and FTS3 support in iOS

[Marco Bambini]

4.3 MB seems really too big... you are probably building a debug version of the 
library.

--
Marco Bambini
http://www.sqlabs.com






On May 16, 2011, at 5:01 PM, Tito Ciuro wrote:

> Hello,
> 
> I have a question about SQLite running on iOS. If I'm not mistaken, SQLite on 
> iOS is not compiled with R*Tree and FTS3. Compiling a static library of 
> SQLite's amalgamated version weighs at about 4.3 MB, which represents almost 
> 25% of the 20 MB-per-app allowed on the App Store. For many, this is a major 
> setback because many apps can easily reach this limit.
> 
> My question is: since a "light" version of SQLite is already included in iOS, 
> would it be too complicated to build a static library with only R*Tree and 
> FTS3 support? The idea being of course that the app would link against iOS' 
> SQLite and the app's R*Tree/FTS3 library, thus reducing the app's footprint 
> considerably.
> 
> Are there dependencies that would make this attempt a nightmare? Has anyone 
> gone through this?
> 
> Thanks in advance,
> 
> -- Tito
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users