Re: Command line utilities for tab-separated value files
On Wednesday, 13 April 2016 at 16:34:16 UTC, Jon D wrote: On Wednesday, 13 April 2016 at 07:34:11 UTC, Rory McGuire wrote: On Wed, Apr 13, 2016 at 3:41 AM, Puming via Digitalmars-d-announce < digitalmars-d-announce@puremagic.com> wrote: On Tuesday, 12 April 2016 at 06:22:55 UTC, Puming wrote: Here is what I know of it, using subPackages: Just tried your suggestion and it works. I just added the below to the parent project to get the apps build: void main() { import std.process : executeShell; executeShell(`dub build :app1`); executeShell(`dub build :app2`); executeShell(`dub build :app3`); } Thanks Rory, Puming. I'll look into this ... Available now via DUB. Setup follows the outline Rory and Puming suggested, plus a few other changes. Only useful for people already using DUB, but seems convenient and does get it included in the package registry. Works as follows: $ dub fetch tsv-utils-dlang $ dub run tsv-utils-dlang This kicks off a build of the package, binaries are in the DUB package repository. The user has to add them to the PATH.
Re: Command line utilities for tab-separated value files
On Wednesday, 13 April 2016 at 19:52:30 UTC, Walter Bright wrote: On 4/11/2016 5:50 PM, Jon D wrote: I'd welcome any feedback, either on the apps or the code. Intention is that the code be reasonable example programs. And, I may write a blog post about my D explorations at some point, they'd be referenced in such an article. You've got questions on: https://www.reddit.com/r/programming/comments/4ems6a/commandline_utilities_for_large_tabseparated/ !! As the author, it'd be nice to do an AMA there. Thanks for posting there and letting me know. I responded and will watch the thread. What do you mean by an "AMA"?
Re: Command line utilities for tab-separated value files
On Wednesday, 13 April 2016 at 18:22:21 UTC, Dicebot wrote: On Wednesday, 13 April 2016 at 17:21:58 UTC, Jon D wrote: You don't need to put anything on path to run utils from dub packages. `dub run` will take care of setting necessary envionment (without messing with the system): dub fetch package_with_apps dub run package_with_apps:app1 --flags args These are command line utilities, along the lines of unix 'cut', 'grep', etc, intended to be used as part of unix pipeline. It'd be less convenient to be invoking them via dub. They really should be on the path themselves. Sure, that would be beyond dub scope though. Making binary packages is independent of build system or source layout (and is highly platform-specific). The `dun run` feature is mostly helpful when you need to use one such tool as part of a build process for another dub package. Right. So, partly what I'm wondering is if during the normal dub fetch/run cycle there might be an opportunity to print a message the user with some info to help them add the tools to their path. I haven't used dub much, so I'll have to look into it more. But there should be some way to make it reasonably easy and clear. It'll probably be a few days before I can get to this, but I would like to get them in the package registry. --Jon
Re: Command line utilities for tab-separated value files
On Wednesday, 13 April 2016 at 17:01:33 UTC, Dicebot wrote: On Wednesday, 13 April 2016 at 16:34:16 UTC, Jon D wrote: Thanks Rory, Puming. I'll look into this and see how best to make it fit. I'm realizing also there's one additional capability it'd be nice to have in dub for tools like this, which in an option to install the executables somewhere that can be easily be put on the path. Still, even without this there'd be benefit to having them fetched via dub. You don't need to put anything on path to run utils from dub packages. `dub run` will take care of setting necessary envionment (without messing with the system): dub fetch package_with_apps dub run package_with_apps:app1 --flags args These are command line utilities, along the lines of unix 'cut', 'grep', etc, intended to be used as part of unix pipeline. It'd be less convenient to be invoking them via dub. They really should be on the path themselves. --Jon
Re: Command line utilities for tab-separated value files
On Wednesday, 13 April 2016 at 12:36:56 UTC, Dejan Lekic wrote: On Tuesday, 12 April 2016 at 00:50:24 UTC, Jon D wrote: I've open sourced a set of command line utilities for manipulating tab-separated value files. I rarely need TSV files, but I deal with CSV files every day. - It would be nice to test your implementation against std.csv (it can use TAB as separator). Did you try to compare the two? No, I didn't try using the std.csv library utilities. The utilities all take a delimiter, so comma can be specified, but that won't handle CSV escaping. For myself, I'd be more inclined to add TSV-CSV converters rather than adding native CSV support to each tool, but if you're working with CSV all the time that'd be nuisance. If you want, you can try rewriting the inner loop of one of the tools to use csvNextToken rather than algorithm.splitter. tsv-select would be the easiest of the tools to try. It'd also be necessary to replace the writeln for the output to properly add CSV escapes. --Jon
Re: Command line utilities for tab-separated value files
On Wednesday, 13 April 2016 at 07:34:11 UTC, Rory McGuire wrote: On Wed, Apr 13, 2016 at 3:41 AM, Puming via Digitalmars-d-announce < digitalmars-d-announce@puremagic.com> wrote: On Tuesday, 12 April 2016 at 06:22:55 UTC, Puming wrote: Here is what I know of it, using subPackages: Just tried your suggestion and it works. I just added the below to the parent project to get the apps build: void main() { import std.process : executeShell; executeShell(`dub build :app1`); executeShell(`dub build :app2`); executeShell(`dub build :app3`); } Thanks Rory, Puming. I'll look into this and see how best to make it fit. I'm realizing also there's one additional capability it'd be nice to have in dub for tools like this, which in an option to install the executables somewhere that can be easily be put on the path. Still, even without this there'd be benefit to having them fetched via dub. --Jon
Re: Command line utilities for tab-separated value files
On Tuesday, 12 April 2016 at 06:22:55 UTC, Puming wrote: On Tuesday, 12 April 2016 at 00:50:24 UTC, Jon D wrote: Hi all, I've open sourced a set of command line utilities for manipulating tab-separated value files. They are complementary to traditional unix tools like cut, grep, etc. They're useful for manipulating large data files. I use them when prepping files for R and similar tools. These tools were part of my 'explore D' programming exercises. [...] Interesting, I have large csv files, and this lib will be useful. Can you put it onto code.dlang.org so that we could use it with dub? I'd certainly like to make it available via dub, but I wasn't sure how to set it up. There are two issues. One is that the package builds multiple executables, which dub doesn't seem to support easily. More problematic is that quite a bit of the test suite is run against the executables, which I could automate using make, but didn't see how to do it with dub. If there are suggestions for setting this up in dub that'd be great. An example project doing something similar would be really helpful. --Jon
Command line utilities for tab-separated value files
Hi all, I've open sourced a set of command line utilities for manipulating tab-separated value files. They are complementary to traditional unix tools like cut, grep, etc. They're useful for manipulating large data files. I use them when prepping files for R and similar tools. These tools were part of my 'explore D' programming exercises. The tools are here: https://github.com/eBay/tsv-utils-dlang They are likely of interest primarily to people regularly working with large files, though others might find the performance benchmarks of interest as well (included in the README). I'd welcome any feedback, either on the apps or the code. Intention is that the code be reasonable example programs. And, I may write a blog post about my D explorations at some point, they'd be referenced in such an article. --Jon
Re: Weak Purity Blog Post
On Monday, 28 March 2016 at 01:44:02 UTC, sarn wrote: D's implementation of functional purity supports "weak" purity - functions that can mutate arguments but are otherwise traditionally pure. I wrote a post about some of the practical benefits of this kind of purity: https://theartofmachinery.com/2016/03/28/dirtying_pure_functions_can_be_useful.html Nice article. A suggestion: The point about improved testability when designing for purity is well made. In D, this is further supported by the ability to write and place unit tests alongside the functions themselves. That's my personal opinion at least - because unit test are so easy to write in D, it encourages design for testability. My suggestion is to add a note about this to the post. --Jon
Re: Vision for the first semester of 2016
On Monday, 25 January 2016 at 02:37:40 UTC, Andrei Alexandrescu wrote: Hot off the press! http://wiki.dlang.org/Vision/2016H1 -- Andrei A couple comments: a) No mention of targeting increased organizational participation (academic, corporate, etc). Not trying to suggest it should or shouldn't be a goal. Just that if it is goal meaningful effort will be directed toward in H1 then it'd be worth including in the writeup. b) More specificity in the roadmap and priorities, to the extent they are known - As a potential D adopter, it'd be useful to have better insight into where the language might be a year or two out. For example, what forms of C++ integration might be available, or if the major components of the standard library are likely to be available nogc. However, it's hard to discern this from the writeup. Perhaps in many cases it would be premature to establish such goals, but to the extent there has been concrete thought it'd be useful to write it up. This comment is similar to a number of others suggesting a preference for more concrete goals. --Jon
Re: D Article: Memory Safety
On Wednesday, 20 January 2016 at 14:04:53 UTC, Jakob Ovrum wrote: The article aims to explain how to use @safe, @system and importantly, @trusted, including all the hairy details of templates. https://jakobovrum.github.io/d/2016/01/20/memory-safety.html Any and all feedback appreciated. Nice article. I got a much better understanding reading it. A small thing - I immediately tried adding @safe to one of my programs and hit an issue related to the first example in the article: void main() @safe { import std.stdio; writeln("hello, world"); } This is passes the @safe constraint, but 'stdout.writeln()' and 'stderr.writeln()' do not. (My program uses stderr.) stderr/stdout/stdin are __gshared and can't be referenced by safe code. The module level version of writeln, etc., access a trusted version of stdout to avoid this. I don't have a specific suggestion for addressing this in the article. It's nicely written and delving into this may take away from the key points. But, I wanted to point this out in case others hit it. (Note: This related to a recent thread on the learn forum: http://forum.dlang.org/thread/vkihzrwomhiwdzqel...@forum.dlang.org)
Re: Silicon Valley D Meetup December 17, 2015
On Friday, 18 December 2015 at 16:01:48 UTC, Andrei Alexandrescu wrote: On 12/17/2015 10:07 PM, Ali Cehreli wrote: On Thursday, 17 December 2015 at 17:41:30 UTC, Ali Çehreli wrote: On 12/12/2015 05:03 PM, Ali Çehreli wrote: Our guest speaker is Steven Schveighoffer. He will present "Mutability wildcards in D": How was it? -- Andrei From a newcomer's perspective (my 2nd meet-up) - Excellent. Steve's presentation improved my understanding of the language, and the opportunity for discussions with core members of the D community is fantastic. Thanks to Steve, Ali, and Truedat for putting this together. --Jon