[gentoo-user] [OT] awk - sed - grep = tying-myself-up-in-knots

[Mick]

I've been struggling to parse/split/substitute some names and numbers using a 
spreadsheet and think that this task may be easier to achieve using 
conventional *nix tools. The problem is I wouldn't know where to start.

I have a directory with loads of images.  Each image file name has a 
description comprising hyphen-separated words and a part number, also hyphen 
separated; e.g.:

some-description-with-words-012-63099.jpg

The number and length of the words change for each file.  The part number 
always has two components separated by a hyphen, but may also change in length 
and acquire more/fewer digits.

I need two outputs:

1. the description + " (per M²)", like so:

some-description-with-words (per M²)

2. the part number, but replacing the hyphen with "/", like so:

012/63099


I can list the directory contents and redirect all image file names into a txt 
file.  What I am looking for is some additional steps I can pipe it through to 
obtain the two outputs, either in the same file or different files.  These 
file(s) are then imported into a spreadsheet template and manipulated, before 
the result is ultimately exported from the spreadsheet and uploaded to a 
server as a CSV file.

Is this parsing, splitting and substitution exercise achievable?  Any 
suggestions to try out?

-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] [OT] awk - sed - grep = tying-myself-up-in-knots

[Dale]

Mick wrote:
> <<< SNIP >>>
>
> 2. the part number, but replacing the hyphen with "/", like so:
>
> 012/63099
>
>
>

I'm no expert on this sort of thing, and some other subscribers here
could verify this, but I have a question.  Wouldn't using a "/" make it
think of it as a directory instead of a file name?   Or is that the
intention? 

Just tossing that out there.

Dale

:-)  :-) 

Re: [gentoo-user] [OT] awk - sed - grep = tying-myself-up-in-knots

[Nils Freydank]

Hi Mick!

Am Mittwoch, 10. Februar 2016, 17:02:46 CET schrieb Mick:
> I've been struggling to parse/split/substitute some names and numbers using
> a spreadsheet and think that this task may be easier to achieve using
> conventional *nix tools. The problem is I wouldn't know where to start.

I was taught by other linux users (mostly fanatic KISS people from Archlinux 
:D) to use as less different tools and therefor dependencies for a script/
shell command as possible. In your case, simple bash might be even enough:

Check the bash man page for "PARAMETER EXPANSION" (these $
{foo...somethingspecial} instead of $foo).

> I have a directory with loads of images.  Each image file name has a
> description comprising hyphen-separated words and a part number, also hyphen
> separated; e.g.:

> some-description-with-words-012-63099.jpg
> 
> The number and length of the words change for each file.  The part number
> always has two components separated by a hyphen, but may also change in
> length and acquire more/fewer digits.
> 
> I need two outputs:
> 
> 1. the description + " (per M²)", like so:
> 
> some-description-with-words (per M²)

Just save both informations to one variable, then print/echo the value of 
these variables (something like "echo $VAR1 $VAR2" or even "echo $VAR1 (per 
M²)".

> 2. the part number, but replacing the hyphen with "/", like so:
> 
> 012/63099

Again, parameter substitution might work. :-)

> I can list the directory contents and redirect all image file names into a
> txt file.  What I am looking for is some additional steps I can pipe it
> through to obtain the two outputs, either in the same file or different
> files.  These file(s) are then imported into a spreadsheet template and
> manipulated, before the result is ultimately exported from the spreadsheet
> and uploaded to a server as a CSV file.
> 
> Is this parsing, splitting and substitution exercise achievable?  Any
> suggestions to try out?
"Everything is possible"™


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] [OT] awk - sed - grep = tying-myself-up-in-knots

[J. Roeleveld]

On Wednesday, February 10, 2016 05:02:46 PM Mick wrote:
> I've been struggling to parse/split/substitute some names and numbers using
> a spreadsheet and think that this task may be easier to achieve using
> conventional *nix tools. The problem is I wouldn't know where to start.
> 
> I have a directory with loads of images.  Each image file name has a
> description comprising hyphen-separated words and a part number, also hyphen
> separated; e.g.:
> 
> some-description-with-words-012-63099.jpg
> 
> The number and length of the words change for each file.  The part number
> always has two components separated by a hyphen, but may also change in
> length and acquire more/fewer digits.
> 
> I need two outputs:
> 
> 1. the description + " (per M²)", like so:
> 
> some-description-with-words (per M²)
> 
> 2. the part number, but replacing the hyphen with "/", like so:
> 
> 012/63099
> 
> 
> I can list the directory contents and redirect all image file names into a
> txt file.  What I am looking for is some additional steps I can pipe it
> through to obtain the two outputs, either in the same file or different
> files.  These file(s) are then imported into a spreadsheet template and
> manipulated, before the result is ultimately exported from the spreadsheet
> and uploaded to a server as a CSV file.
> 
> Is this parsing, splitting and substitution exercise achievable?  Any
> suggestions to try out?

Does this help:

echo "some-description-with-words-012-63099.jpg" | sed 
's/\(.*\)-\(.*\)-\(.*\).jpg$/desc=\1   (per M²) part= \2\/\3 /'

(All on 1 line)

I get the following as output:

desc=some-description-with-words (per M²)  part= 012/63099


--
Joost


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] [OT] awk - sed - grep = tying-myself-up-in-knots

[Alexander Kapshuk]

On Wed, Feb 10, 2016 at 7:02 PM, Mick  wrote:
> I've been struggling to parse/split/substitute some names and numbers using a
> spreadsheet and think that this task may be easier to achieve using
> conventional *nix tools. The problem is I wouldn't know where to start.
>
> I have a directory with loads of images.  Each image file name has a
> description comprising hyphen-separated words and a part number, also hyphen
> separated; e.g.:
>
> some-description-with-words-012-63099.jpg
>
> The number and length of the words change for each file.  The part number
> always has two components separated by a hyphen, but may also change in length
> and acquire more/fewer digits.
>
> I need two outputs:
>
> 1. the description + " (per M²)", like so:
>
> some-description-with-words (per M²)
>
> 2. the part number, but replacing the hyphen with "/", like so:
>
> 012/63099
>
>
> I can list the directory contents and redirect all image file names into a txt
> file.  What I am looking for is some additional steps I can pipe it through to
> obtain the two outputs, either in the same file or different files.  These
> file(s) are then imported into a spreadsheet template and manipulated, before
> the result is ultimately exported from the spreadsheet and uploaded to a
> server as a CSV file.
>
> Is this parsing, splitting and substitution exercise achievable?  Any
> suggestions to try out?
>
> --
> Regards,
> Mick

To get the desired output in two distinct steps, you could try this:

echo 'some-description-with-words-012-63099.jpg' |
sed 's!-[0-9][0-9]*.*! (per M²)!'
some-description-with-words (per M²)

echo 'some-description-with-words-012-63099.jpg' |
sed 's![^0-9][^0-9]*!!; s![.][^0-9]*!!; s!-!/!'
012/63099

To get both types of output on the same line separated with white
space, you could try this:

echo 'some-description-with-words-012-63099.jpg' |
sed '
H
s!-[0-9][0-9]*.*! (per M²)!
x
s![^0-9][^0-9]*!!
s![.][^0-9]*!!
s!-!/!
H
g
y!\n! !
'
some-description-with-words (per M²) 012/63099

[gentoo-user] Re: [OT] awk - sed - grep = tying-myself-up-in-knots

[James]

Mick  gmail.com> writes:


> some-description-with-words-012-63099.jpg
> The number and length of the words change for each file.  The part number 
> always has two components separated by a hyphen, but may also change 
> in length and acquire more/fewer digits.  I need two outputs:

> 1. the description + " (per M²)", like so:
> some-description-with-words (per M²)

My experience with scripting as to what you exactly need, suggests
a trial an error process. I'd take a sampling of the names of the 
files and post them up somewhere where folks can first test an actual script
against some of the actual filenames as to your specification needs.

https://gist.github.com/

Bash is probably quickest, but it might be great fun to test some python
code too (note, I'm just learning/noodling with python codes, but it
is a quite pleasurable journey tinkering around with python codes.


> 2. the part number, but replacing the hyphen with "/", like so: 
> 012/63099

Here is some /awk/ reference that might give you the ideas you need::

http://www.grymoire.com/Unix/Awk.html#uh-46

There was also a recent thread on gentoo-dev about revision numbering
intricacies that was fascinating to read as to the tricks to sort numbers
that use a variety of details as to what number is next in a sequence. If
one can discern that thread in detail, surely your questions a small subset
of those tricks described or alluded to therein.


hth,
James

Re: [gentoo-user] llvm clang ABI confusion

[Alan McKinnon]

On 10/02/2016 23:52, James wrote:
> Hello,
> 
> My previous post suggest that every file under /etc/portage
> 
> can be parsed for flags and directives. I removed all of the archives
> 
> and that cleanup  of /etc/portage/ is in progress. This question
> 
> is unrelated, as best as I can tell.
> 
> 
> Ok, so I just went to install something and got this ::
> 
> emerge -uDNvtp  =dev-java/sbt-0.13.10_rc2
> 
> [ebuild U ~] dev-java/sbt-0.13.10_rc2::gentoo [0.13.8::gentoo]
> 
> USE="-binary" 210,199 KiB
> 
> [ebuild U ~]  dev-lang/scala-2.11.7-r1:2.11/2.11.7::gentoo
> 
> [2.11.4-r1:2.11/2.11.4::gentoo] USE="source -binary -doc -emacs" 0 KiB
> 
> [ebuild U ~]  dev-lang/scala-2.10.6:2.10/2.10.6::gentoo
> 
> [2.10.4-r2:2.10/2.10.4::gentoo] USE="source -binary -doc -emacs" 82,629 KiB
> 
> Total: 3 packages (3 upgrades), Size of downloads: 292,827 KiB
> 
> WARNING: One or more updates/rebuilds have been skipped due to a 
> dependency conflict:
> 
> sys-devel/llvm:0
> 
>   sys-devel/llvm-3.5.0:0/3.5::gentoo, ebuild scheduled for merge
> conflicts with
>
> sys-devel/llvm-3.5.0[clang(-),-debug,python,static-
> analyzer,abi_x86_32(-),abi_x86_64(-)]
> 
> required by (sys-devel/clang-3.5.0-r100:0/3.5::gentoo, installed) 
> 
> I have both installed::   [I] sys-devel/llvm
> 
>  Installed versions:  3.5.0(06:15:37 PM 01/25/2016)(clang libffi ncurses
> 
> python 
> static-analyzer -debug -doc -gold -libedit -multitarget -ocaml -test -xml 
> 
> ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" 
> 
> ABI_X86="32 64 -x32" KERNEL="-Darwin -FreeBSD" PYTHON_TARGETS="python2_7 
> -pypy" 
> VIDEO_CARDS="radeon")
> 
> and  [I] sys-devel/clang
>  Installed versions:  3.5.0-r100(12:07:02 PM 01/17/2016)(python
> 
> static-analyzer -debug -multitarget ABI_MIPS="-n32 -n64 -o32" 
> 
> ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")
> 
> 
> in my /etc/portage/package.use/000.abi  I have this 
> entry::   */* -abi_x86_32
> 
> 
> both "egrep -r clang /etc/portage/" and "egrep -r clang /etc/portage/"
> return empty. Commenting out the above referenced entry in did not fix the
> conflict. 
> 
> I still want to ensure I have no 32 bit  packages on this system.
> Is that my problem that clang/llvm require 32 bit sources? If so, how do I
> fix this and keep 32 bit packages off the system? Surely I am confused as to
> resolution options.

The conflict is very simple.

The ebuild says "must have 32 bits!"
You said "no zero zip nada 32 bits!'
The system says "screw this, you figure it out dude, I give up"

Solutions: There are 2, and they are mutually exclusive

1. Relax your stance and accept that some software out there that you
might want is 32 bits
2. Refuse to have 32 bits, so give up on llvm and clang. Find something
else and move on.

You must pick one of those two. There is no magic hidden solution that
magically lets you have something and also not have it.



-- 
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] [OT] awk - sed - grep = tying-myself-up-in-knots

[Mick]

On Wednesday 10 Feb 2016 20:28:29 Alexander Kapshuk wrote:
> On Wed, Feb 10, 2016 at 7:02 PM, Mick  wrote:
> > I've been struggling to parse/split/substitute some names and numbers
> > using a spreadsheet and think that this task may be easier to achieve
> > using conventional *nix tools. The problem is I wouldn't know where to
> > start.
> > 
> > I have a directory with loads of images.  Each image file name has a
> > description comprising hyphen-separated words and a part number, also
> > hyphen separated; e.g.:
> > 
> > some-description-with-words-012-63099.jpg
> > 
> > The number and length of the words change for each file.  The part number
> > always has two components separated by a hyphen, but may also change in
> > length and acquire more/fewer digits.
> > 
> > I need two outputs:
> > 
> > 1. the description + " (per M²)", like so:
> > 
> > some-description-with-words (per M²)
> > 
> > 2. the part number, but replacing the hyphen with "/", like so:
> > 
> > 012/63099
> > 
> > 
> > I can list the directory contents and redirect all image file names into a
> > txt file.  What I am looking for is some additional steps I can pipe it
> > through to obtain the two outputs, either in the same file or different
> > files.  These file(s) are then imported into a spreadsheet template and
> > manipulated, before the result is ultimately exported from the
> > spreadsheet and uploaded to a server as a CSV file.
> > 
> > Is this parsing, splitting and substitution exercise achievable?  Any
> > suggestions to try out?
> > 
> > --
> > Regards,
> > Mick
> 
> To get the desired output in two distinct steps, you could try this:
> 
> echo 'some-description-with-words-012-63099.jpg' |
> sed 's!-[0-9][0-9]*.*! (per M²)!'
> some-description-with-words (per M²)
> 
> echo 'some-description-with-words-012-63099.jpg' |
> sed 's![^0-9][^0-9]*!!; s![.][^0-9]*!!; s!-!/!'
> 012/63099
> 
> To get both types of output on the same line separated with white
> space, you could try this:
> 
> echo 'some-description-with-words-012-63099.jpg' |
> sed '
> H
> s!-[0-9][0-9]*.*! (per M²)!
> x
> s![^0-9][^0-9]*!!
> s![.][^0-9]*!!
> s!-!/!
> H
> g
> y!\n! !
> '
> some-description-with-words (per M²) 012/63099


Thank you all so much for your responses!  Plenty to try out.

I love this M/L!  :-)

-- 
Regards,
Mick

signature.asc
Description: This is a digitally signed message part.

[gentoo-user] llvm clang ABI confusion

[James]

Hello,

My previous post suggest that every file under /etc/portage

can be parsed for flags and directives. I removed all of the archives

and that cleanup  of /etc/portage/ is in progress. This question

is unrelated, as best as I can tell.


Ok, so I just went to install something and got this ::

emerge -uDNvtp  =dev-java/sbt-0.13.10_rc2

[ebuild U ~] dev-java/sbt-0.13.10_rc2::gentoo [0.13.8::gentoo]

USE="-binary" 210,199 KiB

[ebuild U ~]  dev-lang/scala-2.11.7-r1:2.11/2.11.7::gentoo

[2.11.4-r1:2.11/2.11.4::gentoo] USE="source -binary -doc -emacs" 0 KiB

[ebuild U ~]  dev-lang/scala-2.10.6:2.10/2.10.6::gentoo

[2.10.4-r2:2.10/2.10.4::gentoo] USE="source -binary -doc -emacs" 82,629 KiB

Total: 3 packages (3 upgrades), Size of downloads: 292,827 KiB

WARNING: One or more updates/rebuilds have been skipped due to a 
dependency conflict:

sys-devel/llvm:0

  sys-devel/llvm-3.5.0:0/3.5::gentoo, ebuild scheduled for merge
conflicts with
   
sys-devel/llvm-3.5.0[clang(-),-debug,python,static-
analyzer,abi_x86_32(-),abi_x86_64(-)]

required by (sys-devel/clang-3.5.0-r100:0/3.5::gentoo, installed) 

I have both installed::   [I] sys-devel/llvm

 Installed versions:  3.5.0(06:15:37 PM 01/25/2016)(clang libffi ncurses

python 
static-analyzer -debug -doc -gold -libedit -multitarget -ocaml -test -xml 

ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" 

ABI_X86="32 64 -x32" KERNEL="-Darwin -FreeBSD" PYTHON_TARGETS="python2_7 -pypy" 
VIDEO_CARDS="radeon")

and  [I] sys-devel/clang
 Installed versions:  3.5.0-r100(12:07:02 PM 01/17/2016)(python

static-analyzer -debug -multitarget ABI_MIPS="-n32 -n64 -o32" 

ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="32 64 -x32")


in my /etc/portage/package.use/000.abi  I have this 
entry::   */* -abi_x86_32


both "egrep -r clang /etc/portage/" and "egrep -r clang /etc/portage/"
return empty. Commenting out the above referenced entry in did not fix the
conflict. 

I still want to ensure I have no 32 bit  packages on this system.
Is that my problem that clang/llvm require 32 bit sources? If so, how do I
fix this and keep 32 bit packages off the system? Surely I am confused as to
resolution options.


Suggestions?
James

Re: [gentoo-user] llvm clang ABI confusion

[Neil Bothwick]

On Wed, 10 Feb 2016 21:52:16 + (UTC), James wrote:

> in my /etc/portage/package.use/000.abi  I have this 
> entry::   */* -abi_x86_32

When package.use (and other package.*) is a directory, the files in it
are concatenated, in lexical order. So your entry appears right at the
start. This means it can be overridden by any of the other files.

> I still want to ensure I have no 32 bit  packages on this system.

Why? What is special about 32 bit libraries? They are dependencies like
any other, if a package needs them why try to stop portage installing
them?


-- 
Neil Bothwick

Never get into fights with ugly people because they have nothing to lose.


pgpR7al99VJE9.pgp
Description: OpenPGP digital signature

[gentoo-user] Re: llvm clang ABI confusion

[James]

James  tampabay.rr.com> writes:


> > 1. Relax your stance and accept that some software out there that you
> > might want is 32 bits
> > 2. Refuse to have 32 bits, so give up on llvm and clang. Find something
> > else and move on.

> > You must pick one of those two. There is no magic hidden solution that
> > magically lets you have something and also not have it.

> OK, (1) can be moved to another system or in a container for those
> experiments. I'll choose door number (2).

oops transposed the response  ==>> choose option 1. 

Option 2 
is for minimalist 
cluster systems testing.

James

[gentoo-user] Re: llvm clang ABI confusion

[James]

Alan McKinnon  gmail.com> writes:


> 1. Relax your stance and accept that some software out there that you
> might want is 32 bits
> 2. Refuse to have 32 bits, so give up on llvm and clang. Find something
> else and move on.

> You must pick one of those two. There is no magic hidden solution that
> magically lets you have something and also not have it.


OK, (1) can be moved to another system or in a container for those
experiments. I'll choose door number (2).

thx.
James

[gentoo-user] Re: llvm clang ABI confusion

[James]

Neil Bothwick  digimed.co.uk> writes:


> > I still want to ensure I have no 32 bit  packages on this system.

> Why? What is special about 32 bit libraries? They are dependencies like
> any other, if a package needs them why try to stop portage installing
> them?

Experimentation. I trying to only use 64 bit for minimalistic systems
development for both embedded and larger linux systems. The idea it to
be able to aggregate a variety of 64 bit only systems into clusters. Not
really worth going into in depth. Of the 33 packages that recompiled with
32 bit requirements, none are critical to the cluster development work,
so I'll just move that work onto a more appropriate system(s).


thx,
James