Re: [Discuss] Please help with a BASH puzzle
On 8/30/19 9:50 PM, Dale R. Worley wrote: discuss-requ...@blu.org writes: From: Bill Horne I'm trying to do something that should be very easy to do, and yet I can't remember how to do it, and I'm asking for help. I have an alpha-numeric, sorted file, that looks like this: I'm trying to remember what BASH utility, script, or? command would flag the missing value (in this case, "01AD"). Of course, the hard part is doing it without materializing the list of all possible words with which you want to compare your file. Dale Well, I already showed two different ways of doing that, with brace expansions in bash and with bc, which can do base 36 math to calculate the next expected value. ___ Discuss mailing list Discuss@blu.org http://lists.blu.org/mailman/listinfo/discuss
Re: [Discuss] Please help with a BASH puzzle
discuss-requ...@blu.org writes:
> From: Bill Horne
> I'm trying to do something that should be very easy to do, and yet I
> can't remember how to do it, and I'm asking for help.
>
> I have an alpha-numeric, sorted file, that looks like this:
> I'm trying to remember what BASH utility, script, or? command would flag
> the missing value (in this case, "01AD").
Of course, the hard part is doing it without materializing the list of
all possible words with which you want to compare your file.
# Input a series of lines containing one word each and output each word
# extended in all possible ways by one character out of an ordered alphabet.
# In this example, the alphabet is 0, 1, 2, and 3.
function extend () {
while read WORD
do
echo ${WORD}0
echo ${WORD}1
echo ${WORD}2
echo ${WORD}3
done
}
# Compare the complete list of 3-letter words with the file "sample".
comm -13 sample <( echo | extend | extend | extend )
Dale
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
Re: [Discuss] Please help with a BASH puzzle
On Fri, 23 Aug 2019 23:03:26 -0400 Chris Tyler wrote: > ...Which you can then compare your input file against using the comm > command. (It runs in <5s on my chromebook here). comm is fine, and it's faster than diff if the files are small enough to fit in RAM. You'll have problems if comm starts paging which is why I favor diff as a general purpose solution. -- Rich Pieri ___ Discuss mailing list Discuss@blu.org http://lists.blu.org/mailman/listinfo/discuss
Re: [Discuss] Please help with a BASH puzzle
On Fri, Aug 23, 2019 at 11:05 PM Chris Tyler wrote:
> This will generate a "complete" file (36^4 = 1.6 million lines, 8.1 MB) in
> bash:
>
>a="{$(echo {0..9} {A..Z}|tr " " ,)}"; eval echo $a$a$a$a | tr " " "\012"
> >/tmp/all
>
> ...Which you can then compare your input file against using the comm
> command. (It runs in <5s on my chromebook here).
... and if you don't want to save it to disk, you can wrap it to stream
both inputs on pipe-like file-descriptors as previously described -
(export LC_ALL=C; comm -23 <( a="{$(echo {0..9} {A..Z}|tr " " ,)}"; eval
echo $a$a$a$a | tr " " "\012") $inputfile)
(8MB isn't a large file these days ... but if the real data is wider ...
and thus both files exponentially larger ... might want a more streaming
generator )
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
Re: [Discuss] Please help with a BASH puzzle
This will generate a "complete" file (36^4 = 1.6 million lines, 8.1 MB) in
bash:
a="{$(echo {0..9} {A..Z}|tr " " ,)}"; eval echo $a$a$a$a | tr " " "\012"
>/tmp/all
...Which you can then compare your input file against using the comm
command. (It runs in <5s on my chromebook here).
-Chris
On Fri, Aug 23, 2019 at 9:12 PM David Kramer wrote:
> I should mention I learned this technique from
>
> https://www.cyberciti.biz/faq/explain-brace-expansion-in-cp-mv-bash-shell-commands/
>
> On 8/23/19 9:11 PM, David Kramer wrote:
> > rIs it base 36 then?
> >
> > One way to build the list of expected values is bash brace expansion,
> > where {a,b,c}{1,2,2} will output all combinations of those sets.
> > Here's an example going from 1 to 4 in 4 digits, but you can see how
> > you would do this with 0-9 then A-Z in each of the 4 sets:
> >
> > $ echo {1,2,3,4}{1,2,3,4}{1,2,3,4}{1,2,3,4}
> > 1112 1113 1114 1121 1122 1123 1124 1131 1132 1133 1134 1141 1142
> > 1143 1144 1211 1212 1213 1214 1221 1222 1223 1224 1231 1232 1233 1234
> > 1241 1242 1243 1244 1311 1312 1313 1314 1321 1322 1323 1324 1331 1332
> > 1333 1334 1341 1342 1343 1344 1411 1412 1413 1414 1421 1422 1423 1424
> > 1431 1432 1433 1434 1441 1442 1443 1444 2111 2112 2113 2114 2121 2122
> > 2123 2124 2131 2132 2133 2134 2141 2142 2143 2144 2211 2212 2213 2214
> > 2221 2223 2224 2231 2232 2233 2234 2241 2242 2243 2244 2311 2312
> > 2313 2314 2321 2322 2323 2324 2331 2332 2333 2334 2341 2342 2343 2344
> > 2411 2412 2413 2414 2421 2422 2423 2424 2431 2432 2433 2434 2441 2442
> > 2443 2444 3111 3112 3113 3114 3121 3122 3123 3124 3131 3132 3133 3134
> > 3141 3142 3143 3144 3211 3212 3213 3214 3221 3222 3223 3224 3231 3232
> > 3233 3234 3241 3242 3243 3244 3311 3312 3313 3314 3321 3322 3323 3324
> > 3331 3332 3334 3341 3342 3343 3344 3411 3412 3413 3414 3421 3422
> > 3423 3424 3431 3432 3433 3434 3441 3442 3443 3444 4111 4112 4113 4114
> > 4121 4122 4123 4124 4131 4132 4133 4134 4141 4142 4143 4144 4211 4212
> > 4213 4214 4221 4222 4223 4224 4231 4232 4233 4234 4241 4242 4243 4244
> > 4311 4312 4313 4314 4321 4322 4323 4324 4331 4332 4333 4334 4341 4342
> > 4343 4344 4411 4412 4413 4414 4421 4422 4423 4424 4431 4432 4433 4434
> > 4441 4442 4443
> >
> > I do like Rich's idea of creating the ideal file once, possibly using
> > this technique or really in any programming language, and just doing a
> > diff. If you don't want to do that, though, you could use this
> > technique in an outer for loop, and for each one, read a line from the
> > file under test, and if the line doesn't match the expected value,
> > report the error and loop until you see the next match.
> >
> > On 8/23/19 7:17 PM, Bill Horne wrote:
> >> David,
> >>
> >> Thank you for your suggestion, but they are not only number, but
> >> alpha values too.
> >>
> >> Sorry, though: I chose poor examples. The file can contain any digit
> >> 0-9, and any alpha A-Z. There are no punctuation marks and no white
> >> space.
> >>
> >> Sorted values range from to .
> >>
> >> HTH.
> >>
> >> Bill
> >>
> >>
> >> On 8/23/2019 6:51 PM, David Kramer wrote:
> >>> Are these hex numbers? bc can convert hex to decimal and do hex
> >>> math. The hard part is calculating the next value, and here's an
> >>> example of doing that.
> >>>
> >>> nextValue=`echo "obase=16; ibase=16; ${lastValue} + 1" | bc`
> >>>
> >>> Then all you need to do is compare whether the next line you read is
> >>> equal to nextValue.
> >>>
> >>>
> >>> On 8/23/19 6:32 PM, Bill Horne wrote:
> Thanks for reading this: I appreciate your time.
>
> I'm trying to do something that should be very easy to do, and yet
> I can't remember how to do it, and I'm asking for help.
>
> I have an alpha-numeric, sorted file, that looks like this:
>
> 01AA
>
> 01AB
>
> 01AC
>
> 01AE
>
> 01AF
>
> .. etc.
>
> I'm trying to remember what BASH utility, script, or command would
> flag the missing value (in this case, "01AD"). There are, of
> course, any number of ways to program a solution, but I can't
> remember which of the BASH utilities will do it. All suggestion
> welcome, and thanks in advance.
>
> Bill Horne
>
> ___
> Discuss mailing list
> Discuss@blu.org
> http://lists.blu.org/mailman/listinfo/discuss
> >>> ___
> >>> Discuss mailing list
> >>> Discuss@blu.org
> >>> http://lists.blu.org/mailman/listinfo/discuss
> >> ___
> >> Discuss mailing list
> >> Discuss@blu.org
> >> http://lists.blu.org/mailman/listinfo/discuss
> ___
> Discuss mailing list
> Discuss@blu.org
> http://lists.blu.org/mailman/listinfo/discuss
>
___
Discuss mailing list
Discuss@blu.org
Re: [Discuss] Please help with a BASH puzzle
I should mention I learned this technique from
https://www.cyberciti.biz/faq/explain-brace-expansion-in-cp-mv-bash-shell-commands/
On 8/23/19 9:11 PM, David Kramer wrote:
rIs it base 36 then?
One way to build the list of expected values is bash brace expansion,
where {a,b,c}{1,2,2} will output all combinations of those sets.
Here's an example going from 1 to 4 in 4 digits, but you can see how
you would do this with 0-9 then A-Z in each of the 4 sets:
$ echo {1,2,3,4}{1,2,3,4}{1,2,3,4}{1,2,3,4}
1112 1113 1114 1121 1122 1123 1124 1131 1132 1133 1134 1141 1142
1143 1144 1211 1212 1213 1214 1221 1222 1223 1224 1231 1232 1233 1234
1241 1242 1243 1244 1311 1312 1313 1314 1321 1322 1323 1324 1331 1332
1333 1334 1341 1342 1343 1344 1411 1412 1413 1414 1421 1422 1423 1424
1431 1432 1433 1434 1441 1442 1443 1444 2111 2112 2113 2114 2121 2122
2123 2124 2131 2132 2133 2134 2141 2142 2143 2144 2211 2212 2213 2214
2221 2223 2224 2231 2232 2233 2234 2241 2242 2243 2244 2311 2312
2313 2314 2321 2322 2323 2324 2331 2332 2333 2334 2341 2342 2343 2344
2411 2412 2413 2414 2421 2422 2423 2424 2431 2432 2433 2434 2441 2442
2443 2444 3111 3112 3113 3114 3121 3122 3123 3124 3131 3132 3133 3134
3141 3142 3143 3144 3211 3212 3213 3214 3221 3222 3223 3224 3231 3232
3233 3234 3241 3242 3243 3244 3311 3312 3313 3314 3321 3322 3323 3324
3331 3332 3334 3341 3342 3343 3344 3411 3412 3413 3414 3421 3422
3423 3424 3431 3432 3433 3434 3441 3442 3443 3444 4111 4112 4113 4114
4121 4122 4123 4124 4131 4132 4133 4134 4141 4142 4143 4144 4211 4212
4213 4214 4221 4222 4223 4224 4231 4232 4233 4234 4241 4242 4243 4244
4311 4312 4313 4314 4321 4322 4323 4324 4331 4332 4333 4334 4341 4342
4343 4344 4411 4412 4413 4414 4421 4422 4423 4424 4431 4432 4433 4434
4441 4442 4443
I do like Rich's idea of creating the ideal file once, possibly using
this technique or really in any programming language, and just doing a
diff. If you don't want to do that, though, you could use this
technique in an outer for loop, and for each one, read a line from the
file under test, and if the line doesn't match the expected value,
report the error and loop until you see the next match.
On 8/23/19 7:17 PM, Bill Horne wrote:
David,
Thank you for your suggestion, but they are not only number, but
alpha values too.
Sorry, though: I chose poor examples. The file can contain any digit
0-9, and any alpha A-Z. There are no punctuation marks and no white
space.
Sorted values range from to .
HTH.
Bill
On 8/23/2019 6:51 PM, David Kramer wrote:
Are these hex numbers? bc can convert hex to decimal and do hex
math. The hard part is calculating the next value, and here's an
example of doing that.
nextValue=`echo "obase=16; ibase=16; ${lastValue} + 1" | bc`
Then all you need to do is compare whether the next line you read is
equal to nextValue.
On 8/23/19 6:32 PM, Bill Horne wrote:
Thanks for reading this: I appreciate your time.
I'm trying to do something that should be very easy to do, and yet
I can't remember how to do it, and I'm asking for help.
I have an alpha-numeric, sorted file, that looks like this:
01AA
01AB
01AC
01AE
01AF
.. etc.
I'm trying to remember what BASH utility, script, or command would
flag the missing value (in this case, "01AD"). There are, of
course, any number of ways to program a solution, but I can't
remember which of the BASH utilities will do it. All suggestion
welcome, and thanks in advance.
Bill Horne
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
Re: [Discuss] Please help with a BASH puzzle
rIs it base 36 then?
One way to build the list of expected values is bash brace expansion,
where {a,b,c}{1,2,2} will output all combinations of those sets. Here's
an example going from 1 to 4 in 4 digits, but you can see how you would
do this with 0-9 then A-Z in each of the 4 sets:
$ echo {1,2,3,4}{1,2,3,4}{1,2,3,4}{1,2,3,4}
1112 1113 1114 1121 1122 1123 1124 1131 1132 1133 1134 1141 1142
1143 1144 1211 1212 1213 1214 1221 1222 1223 1224 1231 1232 1233 1234
1241 1242 1243 1244 1311 1312 1313 1314 1321 1322 1323 1324 1331 1332
1333 1334 1341 1342 1343 1344 1411 1412 1413 1414 1421 1422 1423 1424
1431 1432 1433 1434 1441 1442 1443 1444 2111 2112 2113 2114 2121 2122
2123 2124 2131 2132 2133 2134 2141 2142 2143 2144 2211 2212 2213 2214
2221 2223 2224 2231 2232 2233 2234 2241 2242 2243 2244 2311 2312
2313 2314 2321 2322 2323 2324 2331 2332 2333 2334 2341 2342 2343 2344
2411 2412 2413 2414 2421 2422 2423 2424 2431 2432 2433 2434 2441 2442
2443 2444 3111 3112 3113 3114 3121 3122 3123 3124 3131 3132 3133 3134
3141 3142 3143 3144 3211 3212 3213 3214 3221 3222 3223 3224 3231 3232
3233 3234 3241 3242 3243 3244 3311 3312 3313 3314 3321 3322 3323 3324
3331 3332 3334 3341 3342 3343 3344 3411 3412 3413 3414 3421 3422
3423 3424 3431 3432 3433 3434 3441 3442 3443 3444 4111 4112 4113 4114
4121 4122 4123 4124 4131 4132 4133 4134 4141 4142 4143 4144 4211 4212
4213 4214 4221 4222 4223 4224 4231 4232 4233 4234 4241 4242 4243 4244
4311 4312 4313 4314 4321 4322 4323 4324 4331 4332 4333 4334 4341 4342
4343 4344 4411 4412 4413 4414 4421 4422 4423 4424 4431 4432 4433 4434
4441 4442 4443
I do like Rich's idea of creating the ideal file once, possibly using
this technique or really in any programming language, and just doing a
diff. If you don't want to do that, though, you could use this
technique in an outer for loop, and for each one, read a line from the
file under test, and if the line doesn't match the expected value,
report the error and loop until you see the next match.
On 8/23/19 7:17 PM, Bill Horne wrote:
David,
Thank you for your suggestion, but they are not only number, but alpha
values too.
Sorry, though: I chose poor examples. The file can contain any digit
0-9, and any alpha A-Z. There are no punctuation marks and no white
space.
Sorted values range from to .
HTH.
Bill
On 8/23/2019 6:51 PM, David Kramer wrote:
Are these hex numbers? bc can convert hex to decimal and do hex
math. The hard part is calculating the next value, and here's an
example of doing that.
nextValue=`echo "obase=16; ibase=16; ${lastValue} + 1" | bc`
Then all you need to do is compare whether the next line you read is
equal to nextValue.
On 8/23/19 6:32 PM, Bill Horne wrote:
Thanks for reading this: I appreciate your time.
I'm trying to do something that should be very easy to do, and yet I
can't remember how to do it, and I'm asking for help.
I have an alpha-numeric, sorted file, that looks like this:
01AA
01AB
01AC
01AE
01AF
.. etc.
I'm trying to remember what BASH utility, script, or command would
flag the missing value (in this case, "01AD"). There are, of course,
any number of ways to program a solution, but I can't remember which
of the BASH utilities will do it. All suggestion welcome, and thanks
in advance.
Bill Horne
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
Re: [Discuss] Please help with a BASH puzzle
> > . > > If you know what should be in the file then create a new file with the > correct data Or, itemize everything that could be in the file and use diff to find the differnces. > If files are sorted, comm (1) is more efficient and has different output options than diff (1). If necessary, lc or uc the files. The sort and comm must be wrto the same Locale Collation. LC_ALL=C for both will assure that. An awk, perl, or python generator command can be piped to one of the two inputs of comm in modern bash via <() command syntax. I often put canonicalizing commands in both. ___ Discuss mailing list Discuss@blu.org http://lists.blu.org/mailman/listinfo/discuss
Re: [Discuss] Please help with a BASH puzzle
On Fri, 23 Aug 2019 18:32:59 -0400 Bill Horne wrote: > I'm trying to remember what BASH utility, script, or command would > flag the missing value (in this case, "01AD"). There are, of course, > any number of ways to program a solution, but I can't remember which > of the BASH utilities will do it. All suggestion welcome, and thanks > in advance. If you know what should be in the file then create a new file with the correct data and use diff to find the differnces. There's probably a way to do it with awk. -- Rich Pieri ___ Discuss mailing list Discuss@blu.org http://lists.blu.org/mailman/listinfo/discuss
Re: [Discuss] Please help with a BASH puzzle
David,
Thank you for your suggestion, but they are not only number, but alpha
values too.
Sorry, though: I chose poor examples. The file can contain any digit
0-9, and any alpha A-Z. There are no punctuation marks and no white space.
Sorted values range from to .
HTH.
Bill
On 8/23/2019 6:51 PM, David Kramer wrote:
Are these hex numbers? bc can convert hex to decimal and do hex
math. The hard part is calculating the next value, and here's an
example of doing that.
nextValue=`echo "obase=16; ibase=16; ${lastValue} + 1" | bc`
Then all you need to do is compare whether the next line you read is
equal to nextValue.
On 8/23/19 6:32 PM, Bill Horne wrote:
Thanks for reading this: I appreciate your time.
I'm trying to do something that should be very easy to do, and yet I
can't remember how to do it, and I'm asking for help.
I have an alpha-numeric, sorted file, that looks like this:
01AA
01AB
01AC
01AE
01AF
.. etc.
I'm trying to remember what BASH utility, script, or command would
flag the missing value (in this case, "01AD"). There are, of course,
any number of ways to program a solution, but I can't remember which
of the BASH utilities will do it. All suggestion welcome, and thanks
in advance.
Bill Horne
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
Re: [Discuss] Please help with a BASH puzzle
Are these hex numbers? bc can convert hex to decimal and do hex math.
The hard part is calculating the next value, and here's an example of
doing that.
nextValue=`echo "obase=16; ibase=16; ${lastValue} + 1" | bc`
Then all you need to do is compare whether the next line you read is
equal to nextValue.
On 8/23/19 6:32 PM, Bill Horne wrote:
Thanks for reading this: I appreciate your time.
I'm trying to do something that should be very easy to do, and yet I
can't remember how to do it, and I'm asking for help.
I have an alpha-numeric, sorted file, that looks like this:
01AA
01AB
01AC
01AE
01AF
.. etc.
I'm trying to remember what BASH utility, script, or command would
flag the missing value (in this case, "01AD"). There are, of course,
any number of ways to program a solution, but I can't remember which
of the BASH utilities will do it. All suggestion welcome, and thanks
in advance.
Bill Horne
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
___
Discuss mailing list
Discuss@blu.org
http://lists.blu.org/mailman/listinfo/discuss
[Discuss] Please help with a BASH puzzle
Thanks for reading this: I appreciate your time. I'm trying to do something that should be very easy to do, and yet I can't remember how to do it, and I'm asking for help. I have an alpha-numeric, sorted file, that looks like this: 01AA 01AB 01AC 01AE 01AF .. etc. I'm trying to remember what BASH utility, script, or command would flag the missing value (in this case, "01AD"). There are, of course, any number of ways to program a solution, but I can't remember which of the BASH utilities will do it. All suggestion welcome, and thanks in advance. Bill Horne ___ Discuss mailing list Discuss@blu.org http://lists.blu.org/mailman/listinfo/discuss
