Re: [Pharo-dev] Better management of encoding of environment variables
--- Begin Message --- On Wed, 16 Jan 2019 at 18:37, Sven Van Caekenberghe wrote: > Still, one of the conclusions of previous discussions about the encoding > of environment variables was/is that there is no single correct solution. > OS's are not consistent in how the encoding is done in all (historical) > contexts (like sometimes, > 1 env var defines the encoding to use for others, ouch. That one point nearly made my retract my comment next paragraph, but is there much more complexity? or just a case of utf8<==>appSpecificEncoding rather than ascii<==>appSpecificEncoding ? Sorry if I'm rehashing past discussion (do you have a link?), but considering... * 92% of web pages are UTF8 encoded[1] such that pragmatically UTF8 *is* the standard for text * Strings so pervasive in a system ...would there be an overall benefit to adopt UTF8 as the encoding for Strings consistently provided across the cross-platform vm interface? (i.e. fixing platforms that don't comply to the standard due to their historical baggage) And I found it interesting Microsoft are making some moves towards UTF8 [2]... "With insider build 17035 and the April 2018 update (nominal build 17134) for Windows 10, a "Beta: Use Unicode UTF-8 for worldwide language support" checkbox appeared for setting the locale code page to UTF-8.[a] This allows for calling "narrow" functions, including fopen and SetWindowTextA, with UTF-8 strings. " The approach vm-side could be similar to Section 10 How to do text on Windows [3] with the philosophy of "performing the [conversions] as close to API calls as possible, and never holding the [converted] data." [1] https://w3techs.com/technologies/history_overview/character_encoding/ms/y [2] https://en.wikipedia.org/wiki/Unicode_in_Microsoft_Windows [3] http://utf8everywhere.org/ different applications do different things, and other such nice stuff), and > certainly not across platforms. > > So this is really complex. > > Do we want to hide this in some obscure VM C code that very few people can > see, read, let alone help with ? > > The image side is perfectly capable of dealing with platform differences > in a clean/clear way, and at least we can then use the full power of our > language and our tools. > Big question... Do we currently have primitives of the same name returning different encodings on different platforms? I presume that would be awkward. If the image is handle encoding differences, should separate primitives be used? e.g. utf8GetEnv & utf16getEnv Could I get some feedback on [4] saying... **The Single Most Important Fact About Encodings** If you completely forget everything I just explained, please remember one extremely important fact. It does not make sense to have a string without knowing what encoding it uses. " And so... does our String nowadays require an 'encoding' instance variable such that this is *always* associated? This might remove any need for separate utf8GetEnv & utf16getEnv (if that was even a reasonable idea). cheers -ben [4] https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/ > > On 16 Jan 2019, at 10:59, Guillermo Polito > wrote: > > > > Hi Nicolas, > > > > On Wed, Jan 16, 2019 at 10:25 AM Nicolas Cellier < > nicolas.cellier.aka.n...@gmail.com> wrote: > > IMO, windows VM (and plugins) should do the UCS2 -> UTF8 conversion > because the purpose of a VM is to provide an OS independant façade. > > I made progress recently in this area, but we should finish the > job/test/consolidate. > > > > I'm following your changes for windows from the shadows and I think they > are awesome :). > > > > If someone bypass the VM and use direct windows API thru FFI, then he > takes the responsibility, but uniformity doesn't hurt. > > > > So far we are using FFI for this, as you say we create first > Win32WideStrings from utf8 strings and then we use ffi calls to the *W > functions. > > I don't think we can make it for Pharo7.0.0. The cycle to build, do some > acceptance tests, and then bless a new VM as stable is far too long for our > inminent release :). > > > > But this could be for a 7.1.0, and if you like I can surely give a hand > on this. > > > > Guille > > > --- End Message ---
Re: [Pharo-dev] FFILibrary uniqueInstance
--- Begin Message --- On Mon, 14 Jan 2019 at 17:12, Esteban Lorenzano wrote: > Hi, > > On 14 Jan 2019, at 09:46, Guillermo Polito > wrote: > > Hi Nicolas, > On Fri, Jan 11, 2019 at 12:09 PM Nicolas Cellier < > nicolas.cellier.aka.n...@gmail.com> wrote: > >> Hi all, >> I wanted to know if the moduleName was cached somewhere or recomputed at >> each function call... >> It seems to be recomputed. >> > > Do you mean when using a symbol/string as module like...? > > ioFindSurface: id dispatch: dispPtr handle: handlePtr > self ffiCall: #( bool ioFindSurface(int id, void * dispPtr, int > *handlePtr) ) module: #SurfacePlugin > > >> >> We could have a different scheme: >> moduleName is cached in a class inst var of FFILibrary. >> FFILibrary is then added to startup list and moduleName is reset when >> resuming on a different platform. >> instance side moduleName calls class side moduleName. >> >> Or moduleName becomes an inst var, set during uniqueInstance >> construction, and uniqueInstance is nilled out when resuming. >> >> What do you think? >> > > As I see it, there are two (basic) schemas to do FFI calls using UFFI > - using a library object. Library objects are actual singletons stored in > class instance variables > > FFILibrary class >> uniqueInstance > self = FFILibrary > ifTrue: [ self error: 'I''m an abstract class, use one of my children.' ]. > ^ uniqueInstance ifNil: [ uniqueInstance := super new ] > > - using a symbol/string, then a module is created dynamically > > String >> asFFILibrary > ^ FFIUnknownLibrary name: self > > But, but :) the thing is that ffi calls in UFFI are lazily compiled using > bytecode transformations and there (if I'm not mistaken) the library > instances created are cached on the callout object stored as a literal in > the method. > So there is a cache, but per ffi call. And then, at shutdown, all the > generated methods (bound to a specific external function) are wiped out. > > All this process is of course perfectible and probably a "coup de" > profiling could be nice ^^. > > > Well, when I designed the FFILibrary I first thought: let’s the user > decide if they want to cache or not (because most of the time is not > worthy, since as Guille says it is cached on each method and usually is > straightforward) > Then I thought that maybe I should add a generic cache mechanism, for > those cases where you need to perform complicated locates. But I did not > had the time to do it. > > So, I would say: yes, it would be good to have a cache mechanism, but most > of the time is not needed (and caches are dangerous things so better to > reduce their usage at max). > One thing that was get easily confusing is when when the moduleName is not found it seems to cache that error, so that even when the C-library is installed into the correct location it continues to not find the function. I should have added this to the bug tracker a while ago, but I haven't checked for a while so I don't know if this is still the case. I'll wait your response and if agreeable I'll add it to the P8 bug tracker. cheers -ben > If someone wants to implement that for their library, they can. > If someone wants to implement a generic mechanism and contribute it, it > would be also welcomed :) > > Esteban > > > > HTH, > Guille > > > --- End Message ---
[Pharo-dev] external#senders (was Re: DebugSession>>activePC:)
--- Begin Message --- On Thu, 10 Jan 2019 at 23:51, ducasse via Pharo-dev < pharo-dev@lists.pharo.org> wrote: > Thomas can you integrate such comments in the debugger class comment > > @Eliot thanks for the explanation. > About the method removed, could you please react less negatively? It would > be nice. > I cannot believe that you the guy that know the VM would get stopped to > open a bug entry telling us that isOptimizedBlock > has been removed and it should not. How much time opening a bug entry can > take? Under 1 min I guess. > I'd guess it takes more than 1 minute overall - a few minutes to shift context to open an old Pharo image and a few more open the original method to copy it to Pharo and repeat that for the next ten missing methods, and then having fixed it for yourself, rather than just log a job for someone else, having fixed your own you now repair your pharo repo with Iceberg and submit a commit, and your now off-task by half an hour. Not a great deal of time if that was what you schedule to work on, you but frustrating when dragged off task. The thing is, when is someone is frustrated, without sharing there is no chance to resolve anything, so the frustration doubles and builds up, and unconsciously creeps in relationships and/or leads to a breakdown. Putting it out in the world relieves that pressure and provides the possibility that someone might find a middle path. As always, it is not what is said but how it is said, and personally that seemed okay to me. >> Just because a method is not in the image does not imply it is not in use. It simply means that it is not in use in the base image. As the system gets modularised this issue will only increase. On the flip side, if the rule was "don't touch unused methods", that would block a lot of action around cleaning, minimisation and modulation that are important. Even though those things aren't directly the shiney new tools that make Pharo great, its their philosophy that underpins a lot of the visible Pharo improvements which has facilitated Pharo's growth. That "vision" is why I'm here. The pivot point here the concept of "unused" and perhaps where we can do better. Currently developers have no information available to base their decision on. Requiring developers to query the mail list about every cleaning, minimisation and modulation action would have a freezing effect. For stuff that is image its much easier for developers since: * its "visible" right in front of them * they can make decisions and take action around it * tests can be run So the question is how we can get those things for important modules outside the image? For me, VM is not like any third party app but is very much a *part* of Pharo since its something which helps Pharo itself advance. So lets treat it as such, similar to how we treat other modules like Calypso or Iceberg which happen distributed in-Image. Can we consider the last step of the CI (after packing the CI product) could load a static version of VMMaker? Not that any issues would fail the commit, but just report to bring "visibility" to the table ? Or, once VMMaker is in git, perhaps a separate opensmalltalk-vm CI job could run against each latest Pharo image to report problems? Or to broaden the perspective of "unused" further, we could put broader #senders-info in front of developers so then can make informed decisions at design time rather cleaning up after-the-fact? Projects could register on a central site their list of #senders (generated from some tool) so the Pharo IDE could reference that when asked for #senders - so the information is available to make informed decisions. Clicking on an external#sender could pop up the actual code hosted on github, so developers have even better knowledge for decisions and communication. Of course, creating such a system requires effort in our world of limited resources. But the external#senders register could be a premium service available just to Pharo Association/Consortium members which sets up a bit of a win/win scenario. It helps developers and is an incentive to join up. This is not to guarantee changes won't affect your project's #senders, but just to improve communication around them. So why if marcus removed it inadvertently would you want to make him feel > bad? I don’t want people to feel bad. > We can do mistake. That is an important philosophy, but here is a key thing to be clear on... [ "If you are not open to hearing about mistakes, then you are not open to making mistakes." ] repeat. There is no reason for an individual to feel bad about removing an "unused" method if that is the process. But can the process be improved? cheers -ben --- End Message ---
Re: [Pharo-dev] [ANN] Iceberg v1.4.0
--- Begin Message --- On Wed, 7 Nov 2018 at 17:09, Cyril Ferlicot wrote: > > Hello! > > This week we are releasing the version v1.4.0 of Iceberg. > (https://github.com/pharo-vcs/iceberg/releases/tag/v1.4.0) > > This version is available in the latest Pharo 7. > > This release fixes a bug introduced in v1.3. It also add new features > in the repository view, add some cleaning and also re-introduce a > feature that was lost. > > Thanks to all contributors. > > Enjoy! > > Full changelog: > > Bugfixes > > #1068 'There is no associated repository configured.' warning on right > clicking missing repository > > Features > > #1077 Repository view: Allow to collapse branches/remotes/tags trees > #847 Move tags under remotes in Repository view Ohh, those two are very nice. I'm meant to be working on something else but needed to try those out. That long list of tags had been mildly annoying me for a while, but not enough for me to get around to complaining about it. Glad its fixed already. > #1070 set upstream if missing > > Cleanups > > #1066 Pharo 7: PackageManifest subclasses should be packaged with "Manifest" > #1015 Replace usages of Glamour in the Github Plugin > #1063 > 1061-Introduce-iconNamed-in-IceDefinition-and-IceTipModel-and-remove-all-the-terrible-Smalltalk-ui-icons Cool. I like the way the changes are reported and the semantic versioning. cheers -ben --- End Message ---
