Re: [Interest] Size of libQt5Core in 5.3
Thanks again for the info. Here's where I've gotten to so far. Starting from the last ./configure command line I mentioned, plus -no-feature- for every qfeature.txt item (minus LIBRARY, SETTINGS and REGULAREXPRESSION, since it doesn't build without those), I was at 4.3 MB stripped. Added to the size: 1. From the -no-feature- for every feature list, I pulled out a few more of the -no-feature- things for features we need directly or indirectly. So, these things are not specified to ./configure anymore (and so the feature is included). (removed -no-feature-LIBRARY -no-feature-SETTINGS -no-feature-REGULAREXPRESSION earlier) -no-feature-XMLSTREAM -no-feature-XMLSTREAMREADER -no-feature-XMLSTREAMWRITER -no-feature-TEMPORARYFILE -no-feature-TEXTDATE -no-feature-DATESTRING -no-feature-PROPERTIES -no-feature-TEXTCODEC -no-feature-CODECS -no-feature-SYSTEMSEMAPHORE, -no-feature-SHAREDMEMORY, -no-feature-TRANSLATION -no-feature-HTTP -no-feature-UDPSOCKET -no-feature-NETWORKPROXY -no-feature-SOCKS5 -no-feature-NETWORKINTERFACE -no-feature-NETWORKDISKCACHE -no-feature-BEARERMANAGEMENT -no-feature-LOCALSERVER (I still want to switch over to the -qconfig way, but qconfig is not automatically built in the Linux desktop build, and has errors for me when compiling). I realize not all those affect libQt5Core. At that point, libQt5Core.so.5.3.0 was about 4.8 MB after stripping (up from 4.3 MB). Reduced the size: 1. In qtbase/mkspecs/common/gcc-base.conf, I had previously updated QMAKE_CFLAGS_OPTIMIZE to use -Os but had left QMAKE_CLAGS_OPTIMIZE_FULL to -O3. I then updated QMAKE_CFLAGS_OPTIMIZE_FULL to use -Os. (Save ~0.8 MB) 2. Edit corelib.pro to comment out CONFIG += exceptions (Save ~380 KB) 3. Use the steps that John Layt gave for updating the qtbase/util/local_database stuff for locale data, to generate a qlocale.xml, remove locale items for everything except C and US English, then run qlocalexml2cpp.py. Noted the effect on qtbase/src/corelib/tools/qlocale_data_p.h (Save ~0.7 MB) 4. Have version of qtbase/src/corelib/mimetypes/mime/packages/freedesktop.org.xml without comment xml:lang items sed '/\comment xml:lang/d' freedesktop.org.xml no-lang.freedesktop.org.xml Update qtbase/src/corelib/mimetypes/mimetypes.qrc to reference no-lang.freedesktop.org.xml (Save ~220 KB) After these changes, libQt5Core.so.5.3.0 was 3.0 MB stripped! (3048028 bytes). If I trimmed the mime types down to a single item and took out the DOCTYPE, it went down a little bit more (25KB less). So, that's within striking distance now. Other things I might still try: - Updating the TLD stuff to slim it down. - See if I can safely remove some other things from the locale information. John? (likely subtags, language name list, etc.) - If there's an update that makes -no-feature-REGULAREXPRESSION work, use that. A few other comments: Peter Kümmel wrote: Maybe it is worth to think about a Qt library which really only provides essential stuff. What would such a library contain, when all the but the GUI needs it was removed? QtCore generally has just essential stuff for either UI or non-UI. Of course there's other libraries too that have nice things for both UI and non-UI like QtNetwork, etc. QtCore does have some things typically used for UI that might not be needed for non-UI use, like animations, etc., but there are -no-feature items for those (or most those). We've been using Qt for many things for a few years now. It's a great C++ library in general. We have embedded software without any UI, middleware that runs on desktops or (some) mobile without any direct UI, and then UI pieces as well. The middleware piece has been used in apps that run .NET on Windows or Objective-C on Mac (exposing exported C functions or COM interop/PInvoke, and keeping C++ class use internal to the library). There's other libraries too that are interesting for non-UI use that Qt provides. Thiago Macieira Note that it's configuring something outside of Qt and possibly affecting other applications in your system. It seems like the locale reference by qtbase/util/local_database and the locale classes are self contained. It does use data from the outside, but I'm not sure how it would affect other applications in the system (unless they're using the Qt version we make). Thiago Macieira Why are you even trying to build qttools for your target environment? Just skip the module altogether. That's a good idea. I had run it mainly for qtpaths and lrelease. Peter Kümmel So maybe the question should be, what does an essential 1M Qt lib need to be of any use? I don't know what a target size would be per-se, but the specific list might depend on who you ask. There are some things that might sound like UI related at first, like say QSize QPoint and QRect, but those are actually nice to have for non-UI stuff. I think QtCore largely has essential non-UI common things, but a few things that I don't see us using from QtCore for
[Interest] Two candidate updates for 5.3.x
In trying to reduce the size of Qt for an embedded environment, I've run
into a couple problems that would be best solved by updates to the source.
There was a comment that issues with 5.3.0 RC could be sent to the mailing
list here.
1. If -no-gui is specified, qtdiag tries to build, but encounters the error
Project ERROR: Unknown module(s) in QT: gui-private gui
-- To fix, update qttools/src/src.pro, change
!android:!ios:!qnx:!wince*:!winrt*:SUBDIRS += qtdiag
to
qtHaveModule(gui) {
!android:!ios:!qnx:!wince*:!winrt*:SUBDIRS += qtdiag
}
2. -no-feature-REGULAREXPRESSION does not work. For the following desktop
Linux (Ubuntu 12.04 and 14.04 x64) ./configure and make:
./configure -release -opensource -shared -largefile -confirm-license
-no-feature-REGULAREXPRESSION -verbose
make
The following error is encountered:
g++ -c -include .pch/Qt5Core -pipe -msse2 -mfpmath=sse -pthread
-I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -g -O3
-fvisibility=hidden -fvisibility-inlines-hidden -std=c++0x -Wall -W
-D_REENTRANT -fPIC -DQT_NO_MTDEV -DQT_NO_LIBUDEV -DQT_NO_USING_NAMESPACE
-DQT_BUILD_CORE_LIB -DQT_BUILDING_QT -DQT_NO_CAST_TO_ASCII
-DQT_ASCII_CAST_WARNINGS -DQT_MOC_COMPAT -DQT_USE_QSTRINGBUILDER
-DQT_DEPRECATED_WARNINGS -DQT_DISABLE_DEPRECATED_BEFORE=0x05
-DELF_INTERPRETER=\/lib/ld-linux.so.2\ -DPCRE_HAVE_CONFIG_H -DQT_CORE_LIB
-D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -DQT_NO_DEBUG
-I../../mkspecs/linux-g++ -I. -I../../include -I../../include/QtCore
-I../../include/QtCore/5.3.0 -I../../include/QtCore/5.3.0/QtCore -Iglobal
-I../3rdparty/pcre -I../3rdparty/harfbuzz/src -I../3rdparty/md5
-I../3rdparty/md4 -I../3rdparty/sha3 -I.moc -o .obj/qregularexpression.o
tools/qregularexpression.cpp
tools/qregularexpression.cpp:761: error: incomplete type
'QRegularExpression' used in nested name specifier
tools/qregularexpression.cpp:762: error: expected ',' or ';' before '{'
token
tools/qregularexpression.cpp:786: error: redefinition of 'int
convertToPcreOptions'
tools/qregularexpression.cpp:761: error: 'int convertToPcreOptions'
previously defined here
tools/qregularexpression.cpp:786: error: incomplete type
'QRegularExpression' used in nested name specifier
tools/qregularexpression.cpp:807: error: ISO C++ forbids declaration of
'QRegularExpressionMatchPrivate' with no type
tools/qregularexpression.cpp:807: error: expected ';' before '*' token
tools/qregularexpression.cpp:817: error: 'PatternOptions' in class
'QRegularExpression' does not name a type
tools/qregularexpression.cpp:845: error: 'QRegularExpression::MatchType' has
not been declared
tools/qregularexpression.cpp:846: error: 'QRegularExpression::MatchOptions'
has not been declared
tools/qregularexpression.cpp:851: error: field 'regularExpression' has
incomplete type
tools/qregularexpression.cpp:857: error: 'MatchType' in class
'QRegularExpression' does not name a type
..
Thanks,
-Daniel
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest
[Interest] Size of libQt5Core in 5.3
I have an embedded environment where we use non-UI parts of Qt (Qt Core, Qt Networking) on an ARM host processor. We've been using 4.8.4 for a little while. There's some other changes coming to this code, and we've been looking to move to 5.x with the upcoming 5.3. After working through several issues, I got the release candidate of Qt 5.3 compiled for this processor. At first, libQt5Core.so.5.3.0 was 6.6 MB! With 4.8.4, we were able to get libQtCore4 down to 2.6 MB. So I started putting in -skip, -no-feature, etc. having features we would still use, but other things stripped out. I also had a mkspec use -Os to optimize for size. Using something like this: OPENSSL_LIBS=-L(path for openssl) -lssl -lcrypto ./configure -release -opensource -prefix (path for prefix) -extprefix (path for prefix) -shared -largefile -I (path for jpeg) -L (path for jpeg) -sysroot (path for sysroot) -xplatform linux-arm_v5t_le-g++ -confirm-license -system-zlib -system-libpng -system-libjpeg -openssl-linked -no-accessibility -no-freetype -no-harfbuzz -no-glib -no-gui -no-widgets -no-cups -no-iconv -no-evdev -no-icu -no-fontconfig -no-pch -no-dbus -no-xcb -no-directfb -no-linuxfb -no-kms -no-opengl -nomake examples -nomake tests -skip qtdeclarative -skip qtdoc -skip qtgraphicaleffects -skip qtlocation -skip qtquick1 -skip qtquickcontrols -skip qtscript -skip qttranslations -skip qtwebkit -skip qtwebkit-examples -skip qtx11extras -skip qtxmlpatterns -no-feature-TEXTHTMLPARSER -no-feature-TEXTODFWRITER -no-feature-CSSPARSER -no-feature-CONCURRENT -no-feature-DRAGANDDROP -no-feature-SESSIONMANAGER -no-feature-SHORTCUT -no-feature-ACTION -no-feature-DOM -no-feature-FILESYSTEMMODEL -no-feature-FILESYSTEMWATCHER -no-feature-ITEMVIEWS -no-feature-DIRMODEL -no-feature-STANDARDITEMMODEL -no-feature-PROXYMODEL -no-feature-SORTFILTERPROXYMODEL -no-feature-IDENTITYPROXYMODEL -no-feature-STRINGLISTMODEL -no-feature-LISTVIEW -no-feature-TABLEVIEW -no-feature-TREEVIEW -no-feature-DATAWIDGETMAPPER -no-feature-COLUMNVIEW -no-feature-MOVIE -no-feature-IMAGEFORMAT_PPM -no-feature-IMAGEFORMAT_XBM -no-feature-IMAGEFORMAT_XPM -no-feature-FREETYPE -no-feature-BIG_CODECS -no-feature-ICONV -no-feature-FTP -no-feature-ACCESSIBILITY -no-feature-ANIMATION -no-feature-STATEMACHINE -no-feature-GESTURES -no-feature-DBUS -no-feature-XMLSCHEMA -verbose It went down to 5.3 MB. 5.3 MB was still too bit, so I went through the features from qfeatures.txt, and did -no-feature for every single feature listed. libQt5Core wouldn't compile without LIBRARY, SETTINGS and REGULAREXPRESSION, so I took the -no-feature out for those things. Even with all of that removed, libQt5Core.so.5.3.0 was still 4.3 MB after being stripped. That would have some things removed though that we want. Is there any way to get Qt Core to compile smaller, like under 3MB? Am I missing anything to skip or -no-xyz or -nomake or -no-feature? Why is libQt5Core.so.5.3.0 so much bigger than libQtCore4.so.4.8.4? We may just need to move to 4.8.6 and leave it there instead of jumping onto 5.x. Thanks, -Daniel ___ Interest mailing list Interest@qt-project.org http://lists.qt-project.org/mailman/listinfo/interest
[Interest] Cross platform Web services server
I'm wondering what other people are using these days for providing web services from a server point of view for embedded devices, where the server code is integrated as a shared library and geared more toward providing RESTful services for a limited number of clients on the local network. Something that's also cross-platform. Ideally, you'd be able to have different handlers for different base paths. For example, registering a C callback or handler C++ class/interface or Qt signal/slot for /path1, and if a request came in for /path1/thing1?a=b, then the registered hander could deal with that. SSL integration (with QSsl and friends, or openssl) and authentication support are important too. Thanks! -Daniel ___ Interest mailing list Interest@qt-project.org http://lists.qt-project.org/mailman/listinfo/interest
Re: [Interest] Relashionship between time_t and QDateTime
From
http://msdn.microsoft.com/en-us/library/windows/desktop/ms724290(v=vs.85).aspx
The MSDN library states:
To account for daylight saving time when converting a file time to a local
time, use the following sequence of functions instead of using
FileTimeToLocalFileTime:
FileTimeToSystemTime
SystemTimeToTzSpecificLocalTime
SystemTimeToFileTime
For example:
(incoming fileTimeUtc)
FILETIME fileTimeLocal;
SYSTEMTIME systemTimeUtc = {0}, systemTimeLocal = {0};
::FileTimeToSystemTime(fileTimeUtc, systemTimeUtc);
::SystemTimeToTzSpecificLocalTime(NULL, systemTimeUtc,
systemTimeLocal);
::SystemTimeToFileTime(systemTimeLocal, fileTimeLocal);
// SYSTEMTIME will lose accuracy less than a millisecond. We'll tack
that back on.
fileTimeLocal += (fileTimeUtc % Millisecond);
If this is on Windows and before Vista, then past timezones might not be
correct if the rules are different than the current set of daylight savings
transition rules (only Vista and later has the dynamic timezone stuff that
tracks timezone transition changes in years past). In the QDateTime docs I see
On Windows and Windows CE, where the system doesn't support historical DST
data, historical accuracy is not maintained with respect to DST. I don't
believe that's quite true - Windows support historical DST data if you use the
calls above, and its on Vista or later. You can also use calls like
GetDynamicTimeZoneInformation and GetTimeZoneInformationForYear on Vista and
later for more explicit historical timezone info.
The thing to remember is that time_t and FILETIME and QDateTime (at least
=5.1) all represent a moment in time without regard for timezone. Like
2013-08-27T11:50:00. It's up to you to track which timezone the time_t or
FILETIME you have belongs to. (i.e., 2013-08-27T11:50:00-06:00)
I think the best way to track things if timezones are in the mix is to track
the date/time as one thing, and separately track the timezone offset for that
date/time. Then you can convert reliably between different local times or UTC.
We ended up with our own class for tracking time that does that. (Kind of
like the .NET DateTimeOffset compared to the DateTime class).
-Daniel
-Original Message-
From: interest-bounces+qtmailinglist1=bowensite@qt-project.org
[mailto:interest-bounces+qtmailinglist1=bowensite@qt-project.org] On Behalf
Of Constantin Makshin
Sent: Tuesday, August 27, 2013 11:40 AM
To: Qt Interest
Subject: Re: [Interest] Relashionship between time_t and QDateTime
When converting time from UTC to local time, Windows uses the *current* state
of daylight saving, not one that was active at the time the original timestamp
is pointing to.
On 08/27/2013 01:55 PM, Frank Hemer wrote:
On Tuesday 27 August 2013 10:19:06 Calogero Mauceri wrote:
On 8/26/2013 7:30 PM, Thiago Macieira wrote:
On segunda-feira, 26 de agosto de 2013 17:42:58, Calogero Mauceri wrote:
QDateTime myDateTime = QDateTime::fromTime_t(f_mtime);
The date time printed doing a myDateTime.toString() is
Wed Dec 5 12:36:18 2007
Retrieving the last modified information using QFileInfo, the
result is different
QFileInfo fi(filepath);
QDateTime myDateTime = fi.lastModified();
I get this result
Wed Dec 5 11:36:18 2007
That is there is one hour difference. I guess the difference is due
to the daylight saving management, but I can not understand how
that management is performed.
Note: if I look at the file properties on Windows dialog, the last
modified time is shown as
Wed Dec 5 12:36:18 2007
Ah, Windows...
The problem might be simply a matter of timezones. The timestamps on
files on Windows are not stored with time_t, but with some
Windows-specific data. We get a FILETIME back from Win32.
Anyway, up until Qt 5.2, you cannot trust the output of a QDateTime
with qDebug since it does not include the timezone. You have to
ensure that the
dates you're comparing by text are in the same timezone:
qDebug() dt.toUTC();
Thanks for your reply.
Unfortunately the problem is still there even if I force a toUTC()
conversion for both QDateTime, either the one initialized from time_t
or the one returned by QFileInfo :/.
Similarly
QDateTime dtFromTime_t = QDateTime::fromTime_t(mtime).toUTC();
QDateTime dtFromFileInfo = fi.lastModified().toUTC();
int sec = ABS(dtFromTime.secsTo(dtFromFileInfo)); // sec returned
is 3600
Daylight savings handling in windows is somehow 'weird'.
The timestamps of files change when switching the system time from
summer time to winter time and vice versa.
Frank
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest
Re: [Interest] Relashionship between time_t and QDateTime
One key is in the Remarks section of SystemTimeToTzSpecificLocalTime()'s documentation (http://msdn.microsoft.com/en-us/library/windows/desktop/ms724949(v=vs.85).aspx): The SystemTimeToTzSpecificLocalTime function may calculate the local time incorrectly under the following conditions: - *The time zone uses a different UTC offset for the old and new years*. - The UTC time to be converted and the calculated local time are in different years. Thanks for that. It looks like for Windows 7 and later, there's now a SystemTimeToTzSpecificLocalTimeEx (and TzSpecificLocalTimeToSystemTimeEx) which use the dynamic time zone structure. It doesn't list the caveats that SystemTimeToTzSpecificLocalTime mentions, so perhaps its historical accuracy is better (for when DST has changed the rules for the timezone in the past). It is a little more cumbersome to have those functions only in later versions of Windows - you'd need to LoadLibrary/GetProcAddress to be able to call SystemTimeToTzSpecificLocalTimeEx if you detect Windows 7 or later, and use SystemTimeToTzSpecificLocalTime otherwise (if you try to early bind and just call the function in an if statement, you'd get entry point not found on older operating systems). Thanks, -Daniel ___ Interest mailing list Interest@qt-project.org http://lists.qt-project.org/mailman/listinfo/interest
Re: [Interest] Relashionship between time_t and QDateTime
One other note: And since daylight saving transition dates and time tend to slightly drift from year to year, there's absolutely no guarantees of getting correct time conversion results; in general case 1-hour error is nearly inevitable. The timezone structure on Windows tracks the ST-DT and DT-ST rules for the transition, not an absolute date - like 02:00 on the first Sunday in April (http://msdn.microsoft.com/en-us/library/windows/desktop/ms725481(v=vs.85).a spx). So, if the rules haven't changed, but the absolute dates have from one year to the next, I think you'll still get the right answer if you use SystemTimeToTzSpecificLocalTime. But, when the rules change, like they did in the US in 2007 (http://aa.usno.navy.mil/faq/docs/daylight_time.php), that's when it would be wrong to calculate the local time using the current year's rules for some range of dates, and you'd need SystemTimeToTzSpecificLocalTimeEx. So, to test, you could use a UTC FILETIME for a date/time on March 31, 2006, have your system set to one of the timezones in the US that follows daylight savings, then try to convert that to local time. If it's following the current rules, the conversion will be an hour too early. If it's historically accurate, it will be correct. -Daniel ___ Interest mailing list Interest@qt-project.org http://lists.qt-project.org/mailman/listinfo/interest
Re: [Interest] Heavily Commented Example: Simple Single Frontend with Two BackendsHi,
Let me state for the record that I do not use volatiles for thread
synchronization. But the issue at hand is not whether a volatile
can be used for full-featured thread synchronization, but whether
it can be used by one thread to signal a second looping thread to
quit.
It can. In that restricted scenario, even a non-atomic write would
be sufficient.
(snip)
CPU A writes to the volatile signalling variable, but it writes to its
CPU-specific cache. The thread that's looping runs on CPU B, and
repeatedly reads from the volatile signalling variable, but always
reads it from its own CPU-specific cache. So it never gets the signal
to quit, and potentially loops forever.
Well, that's not exactly how processors work. CPU A will eventually get
to write the data from its cache back to main RAM. And CPU B will
eventually
get to notice that and discard its cache. So the code running on CPU B
will
eventually get to see the new value.
The question is only how long that might take.
Also note that you should not implement a busy-wait loop like that, like
a spinlock. At least on x86, if one CPU is constantly reading from the
same
address, it will prevent the others from writing to it. Since you're
busy- looping, you're spending power.
I'm also quite interested in this topic. There are a handful of places where
I've used a similar pattern where one thread is doing a loop, and reading a
stop flag to stop a while loop, and some other thread can write to that stop
flag. That stop flag is a member variable. The loops where I've used it
with a stop variable are not constantly pounding the CPU unnecessarily. For
example, let's say you have a thread that auto saves any changes on a
regular interval, or one that processes work items if there are any. And
a QWaitCondition is used to wake up to either do work, or stop. But if
there isn't anything to do, it's doing a smart wait. For example:
while(!m_stop)
{
m_lock.lock();
AutoSave();
if(!m_stop)
{
m_waitCondition.wait(m_lock, m_autoSaveInterval);
}
m_lock.unlock();
}
With some other method called from another thread like
bool A::stopAndWait(unsigned long timeoutMs)
{
m_stop = true;
m_waitCondition.wakeAll();
return this-wait(timeoutMs);
}
So, in that case, I've typically declared
volatile bool m_stop;
With the intention of using volatile for don't optimize the read away.
After reading some various links like the Herb Sutter article referenced
earlier, a co-worker believes that volatile in this case is unnecessary.
i.e., just use
bool m_stop;
So, I'd like to understand the possibilities for the best cross-platform
code. Going one time too many is OK.
- Is reading and writing to a bool m_stop (no volatile) without a mutex
locked OK? Or could the read of the member variable m_stop realistically be
optimized away?
- Is reading and writing to a volatile bool m_stop without a mutex locked
OK?
- If no locking is OK, is volatile better for m_stop, or does it not matter
(and just causes a little bit slower execution for the read/write of
m_stop)?
- Is not using volatile on bool m_stop OK, but both reading and writing to
m_stop should be done with a mutex locked?
- If m_stop is only read or written with a mutex locked, could the value
ever be stale that is read (causing one time too many)?
- What if m_stop was QAtomicInt instead of bool?
Thanks!
-Daniel
___
Interest mailing list
Interest@qt-project.org
http://lists.qt-project.org/mailman/listinfo/interest
