Re: [go-nuts] Re: Go implementation of CLP

[Jason E. Aten]

on the off-chance that ISO8601 timestamp parsing is new (unlikely since
you're dealing with logs), I'll point out that parsing them is relatively
easy in Go:

import "time"
const RFC3339NanoNumericTZ = "2006-01-02T15:04:05.9-07:00"
tmstr := "2019-05-23T15:09:57.000-05:00"
tm, err = time.Parse(RFC3339NanoNumericTZ, tmstr)
panicOn(err)

On Sat, Nov 19, 2022 at 3:33 PM Jason E. Aten  wrote:

> On Sat, Nov 19, 2022 at 8:45 AM david lion  wrote:
>
>> ...could you let us know more about what kind of queries you want to
>> run? The open-source C++ code supports wildcard queries but some users
>> prefer grep-like regex, SQL-like boolean expressions, etc. We haven't yet
>> settled on one, especially for a binding that needs to be somewhat stable.
>>
>
> I'm not sure what SeaweedFS needs, so maybe Chris Lu will chime in too.
>
> Usually when I'm going through logs, I can narrow my search down to a time
> span with at least roughly known begin and end points. Then inside that
> span of log entries, well its really anything goes. It could be content
> based search, it could be field = value based search; it just really
> depends on the task at hand. So the only thing that I can say with
> conviction is hopefully you can make it very easy to specify an ISO8601
> begin timestamp (down to the nanosecond) and an ISO8601 endx timestamps
> (down to the nanosecond) so that on the [begin, endx) interval is the only
> thing searched.  The [begin, endx) convention is typical for rolling
> queries that should not overlap, since incrementing both begin timestamp
> and endx timestamp by (endx-begin) for example will partition (in the
> mathematical sense of creating exhaustive and mutually exclusive subsets)
> your data.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAPNEFAbSZK3T3PhB8zo49rjrh2xz31FpyfQEtdOi2f8qeW_bLg%40mail.gmail.com.

Re: [go-nuts] Re: Go implementation of CLP

[Jason E. Aten]

On Sat, Nov 19, 2022 at 8:45 AM david lion  wrote:

> ...could you let us know more about what kind of queries you want to run?
> The open-source C++ code supports wildcard queries but some users prefer
> grep-like regex, SQL-like boolean expressions, etc. We haven't yet settled
> on one, especially for a binding that needs to be somewhat stable.
>

I'm not sure what SeaweedFS needs, so maybe Chris Lu will chime in too.

Usually when I'm going through logs, I can narrow my search down to a time
span with at least roughly known begin and end points. Then inside that
span of log entries, well its really anything goes. It could be content
based search, it could be field = value based search; it just really
depends on the task at hand. So the only thing that I can say with
conviction is hopefully you can make it very easy to specify an ISO8601
begin timestamp (down to the nanosecond) and an ISO8601 endx timestamps
(down to the nanosecond) so that on the [begin, endx) interval is the only
thing searched.  The [begin, endx) convention is typical for rolling
queries that should not overlap, since incrementing both begin timestamp
and endx timestamp by (endx-begin) for example will partition (in the
mathematical sense of creating exhaustive and mutually exclusive subsets)
your data.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAPNEFAawMdN0Egx93B1M5hupnJY-afTX7PrT%2BaNk6UuwaAN0ng%40mail.gmail.com.

Re: [go-nuts] Re: Go implementation of CLP

[david lion]

Ah, the parsing component doesn't include search, but it is a necessary 
part to enable searching compressed logs. CLP's searching of compressed 
logs is enabled by the storage format it uses (as it both compresses and 
indexes the logs). The parsing component is how we extract information from 
the logs necessary for the storage format.

We can write bindings for search as well---we only started with compression 
because that's what we had more requests for. To focus our effort, could 
you let us know more about what kind of queries you want to run? The 
open-source C++ code supports wildcard queries but some users prefer 
grep-like regex, SQL-like boolean expressions, etc. We haven't yet settled 
on one, especially for a binding that needs to be somewhat stable.

On Friday, 18 November 2022 at 20:09:06 UTC-5 Jason E. Aten wrote:

> Sorry for not being familiar with the terminology. I thought the search 
> through compressed logs was the most interesting part. Is that included in 
> the "parsing component"?
>
> On Fri, Nov 18, 2022 at 1:21 PM david lion  wrote:
>
>> At the moment we're working on a Go binding around the parsing component 
>> of CLP.
>> You're right, the current code is all C++, but we'll be moving the 
>> parsing component to be standalone and creating a C API for it.
>>
>> Right now, we're only creating bindings for the parsing component since 
>> it seems to be the common element in the use-cases we have come across, but 
>> we'd love to hear about any use-cases you're interested in and how we could 
>> support them. For example, with SeaweedFS it makes more sense to integrate 
>> most of the CLP logic directly into the filesystem itself.
>>
>> On Sunday, 13 November 2022 at 04:28:12 UTC-5 Jason E. Aten wrote:
>>
>>> CLP looks impressive. 
>>>
>>> Is there is a C API then it would probably be easy to write Go bindings 
>>> to it.
>>>
>>> Since there is some kind of python integration, perhaps a C API is 
>>> already there.
>>> I took a quick look but could only see C++ code; it would have to be an 
>>> actual C (not C++) API for cgo to be usable.
>>>
>>> Usually this is not too difficult to add to a C++ project.
>>>
>>> On Wednesday, November 9, 2022 at 2:14:03 AM UTC-6 Bharghava Varun Ayada 
>>> wrote:
>>>
 I'm exploring CLP for a logging platform and I think either a cgo or 
 native go library might be something that would help us.

 On Tuesday, 4 October 2022 at 22:03:34 UTC+5:30 david...@gmail.com 
 wrote:

> It would be awesome to help SeaweedFS, so I'm eager to hear more about 
> your ideas to try and understand anything we can do.
>
> Sorry if some of my questions are naive. At a high level I'm trying to 
> better understand two topics:
> 1. the use cases (or features) we/CLP can help with
> 2. the technical aspects/limitations that make a pure-go 
> implementation necessary/beneficial
>
> I saw that SeaweedFS currently can automatically compresses certain 
> file types with Gzip (very cool btw).
> I can imagine that augmenting this feature with CLP for log file types 
> could be beneficial.
> Are these the log files you're referring to? Does SeaweedFS itself 
> also produce log files that can benefit from CLP? Perhaps CLP could help 
> with both cases?
>
> As I imagine packaging the current form of CLP with SeaweedFS is 
> awkward, is packaging a main reason for a pure-go implementation?
> Would a library with programmatic API be favourable for integration 
> rather than calling out to another tool/executable (as required in CLP's 
> current form)?
> Would a go package using cgo (to reuse some existing cpp code) be 
> desirable or is there a requirement for strictly go code?
>
> -david
>
> On Sunday, 2 October 2022 at 22:36:38 UTC-4 ChrisLu wrote:
>
>> Thanks! CLP is great! 
>>
>> I am working on a distributed file system, SeaweedFS, 
>> https://github.com/seaweedfs/seaweedfs
>> I am interested in a pure-go implementation to store the log files 
>> more efficiently.
>>
>> Chris
>>
>> On Sun, Oct 2, 2022 at 6:45 PM david lion  wrote:
>>
>>> Hi Chris,
>>>
>>> I'm one of the CLP developers. We'd be interested in hearing your 
>>> use case and any features you'd like implemented in CLP.
>>> As for a Go implementation, are you asking about bindings to 
>>> directly call CLP functionality from a Go program or are you interested 
>>> in 
>>> a complete re-write in Go?
>>>
>>> -david
>>>
>>> On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:
>>>
 Seems there are no Go implementation for Compressed Log Processor 
 (CLP) yet? 

 CLP is a tool capable of losslessly compressing text logs and 
 searching the compressed logs without decompression.

 https://github.com/y-scope/clp

Re: [go-nuts] Re: Go implementation of CLP

[Jason E. Aten]

Sorry for not being familiar with the terminology. I thought the search
through compressed logs was the most interesting part. Is that included in
the "parsing component"?

On Fri, Nov 18, 2022 at 1:21 PM david lion  wrote:

> At the moment we're working on a Go binding around the parsing component
> of CLP.
> You're right, the current code is all C++, but we'll be moving the parsing
> component to be standalone and creating a C API for it.
>
> Right now, we're only creating bindings for the parsing component since it
> seems to be the common element in the use-cases we have come across, but
> we'd love to hear about any use-cases you're interested in and how we could
> support them. For example, with SeaweedFS it makes more sense to integrate
> most of the CLP logic directly into the filesystem itself.
>
> On Sunday, 13 November 2022 at 04:28:12 UTC-5 Jason E. Aten wrote:
>
>> CLP looks impressive.
>>
>> Is there is a C API then it would probably be easy to write Go bindings
>> to it.
>>
>> Since there is some kind of python integration, perhaps a C API is
>> already there.
>> I took a quick look but could only see C++ code; it would have to be an
>> actual C (not C++) API for cgo to be usable.
>>
>> Usually this is not too difficult to add to a C++ project.
>>
>> On Wednesday, November 9, 2022 at 2:14:03 AM UTC-6 Bharghava Varun Ayada
>> wrote:
>>
>>> I'm exploring CLP for a logging platform and I think either a cgo or
>>> native go library might be something that would help us.
>>>
>>> On Tuesday, 4 October 2022 at 22:03:34 UTC+5:30 david...@gmail.com
>>> wrote:
>>>
 It would be awesome to help SeaweedFS, so I'm eager to hear more about
 your ideas to try and understand anything we can do.

 Sorry if some of my questions are naive. At a high level I'm trying to
 better understand two topics:
 1. the use cases (or features) we/CLP can help with
 2. the technical aspects/limitations that make a pure-go implementation
 necessary/beneficial

 I saw that SeaweedFS currently can automatically compresses certain
 file types with Gzip (very cool btw).
 I can imagine that augmenting this feature with CLP for log file types
 could be beneficial.
 Are these the log files you're referring to? Does SeaweedFS itself also
 produce log files that can benefit from CLP? Perhaps CLP could help with
 both cases?

 As I imagine packaging the current form of CLP with SeaweedFS is
 awkward, is packaging a main reason for a pure-go implementation?
 Would a library with programmatic API be favourable for integration
 rather than calling out to another tool/executable (as required in CLP's
 current form)?
 Would a go package using cgo (to reuse some existing cpp code) be
 desirable or is there a requirement for strictly go code?

 -david

 On Sunday, 2 October 2022 at 22:36:38 UTC-4 ChrisLu wrote:

> Thanks! CLP is great!
>
> I am working on a distributed file system, SeaweedFS,
> https://github.com/seaweedfs/seaweedfs
> I am interested in a pure-go implementation to store the log files
> more efficiently.
>
> Chris
>
> On Sun, Oct 2, 2022 at 6:45 PM david lion  wrote:
>
>> Hi Chris,
>>
>> I'm one of the CLP developers. We'd be interested in hearing your use
>> case and any features you'd like implemented in CLP.
>> As for a Go implementation, are you asking about bindings to directly
>> call CLP functionality from a Go program or are you interested in a
>> complete re-write in Go?
>>
>> -david
>>
>> On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:
>>
>>> Seems there are no Go implementation for Compressed Log Processor
>>> (CLP) yet?
>>>
>>> CLP is a tool capable of losslessly compressing text logs and
>>> searching the compressed logs without decompression.
>>>
>>> https://github.com/y-scope/clp
>>>
>>> Chris
>>>
>> --
>> You received this message because you are subscribed to a topic in
>> the Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/golang-nuts/XeDIZfTMlX8/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to
>> golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/0c57905a-1096-4688-b268-93742c48316fn%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/XeDIZfTMlX8/unsubscribe.
> To unsubscribe from this group and all its 

Re: [go-nuts] Re: Go implementation of CLP

[david lion]

At the moment we're working on a Go binding around the parsing component of 
CLP.
You're right, the current code is all C++, but we'll be moving the parsing 
component to be standalone and creating a C API for it.

Right now, we're only creating bindings for the parsing component since it 
seems to be the common element in the use-cases we have come across, but 
we'd love to hear about any use-cases you're interested in and how we could 
support them. For example, with SeaweedFS it makes more sense to integrate 
most of the CLP logic directly into the filesystem itself.

On Sunday, 13 November 2022 at 04:28:12 UTC-5 Jason E. Aten wrote:

> CLP looks impressive. 
>
> Is there is a C API then it would probably be easy to write Go bindings to 
> it.
>
> Since there is some kind of python integration, perhaps a C API is already 
> there.
> I took a quick look but could only see C++ code; it would have to be an 
> actual C (not C++) API for cgo to be usable.
>
> Usually this is not too difficult to add to a C++ project.
>
> On Wednesday, November 9, 2022 at 2:14:03 AM UTC-6 Bharghava Varun Ayada 
> wrote:
>
>> I'm exploring CLP for a logging platform and I think either a cgo or 
>> native go library might be something that would help us.
>>
>> On Tuesday, 4 October 2022 at 22:03:34 UTC+5:30 david...@gmail.com wrote:
>>
>>> It would be awesome to help SeaweedFS, so I'm eager to hear more about 
>>> your ideas to try and understand anything we can do.
>>>
>>> Sorry if some of my questions are naive. At a high level I'm trying to 
>>> better understand two topics:
>>> 1. the use cases (or features) we/CLP can help with
>>> 2. the technical aspects/limitations that make a pure-go implementation 
>>> necessary/beneficial
>>>
>>> I saw that SeaweedFS currently can automatically compresses certain file 
>>> types with Gzip (very cool btw).
>>> I can imagine that augmenting this feature with CLP for log file types 
>>> could be beneficial.
>>> Are these the log files you're referring to? Does SeaweedFS itself also 
>>> produce log files that can benefit from CLP? Perhaps CLP could help with 
>>> both cases?
>>>
>>> As I imagine packaging the current form of CLP with SeaweedFS is 
>>> awkward, is packaging a main reason for a pure-go implementation?
>>> Would a library with programmatic API be favourable for integration 
>>> rather than calling out to another tool/executable (as required in CLP's 
>>> current form)?
>>> Would a go package using cgo (to reuse some existing cpp code) be 
>>> desirable or is there a requirement for strictly go code?
>>>
>>> -david
>>>
>>> On Sunday, 2 October 2022 at 22:36:38 UTC-4 ChrisLu wrote:
>>>
 Thanks! CLP is great! 

 I am working on a distributed file system, SeaweedFS, 
 https://github.com/seaweedfs/seaweedfs
 I am interested in a pure-go implementation to store the log files more 
 efficiently.

 Chris

 On Sun, Oct 2, 2022 at 6:45 PM david lion  wrote:

> Hi Chris,
>
> I'm one of the CLP developers. We'd be interested in hearing your use 
> case and any features you'd like implemented in CLP.
> As for a Go implementation, are you asking about bindings to directly 
> call CLP functionality from a Go program or are you interested in a 
> complete re-write in Go?
>
> -david
>
> On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:
>
>> Seems there are no Go implementation for Compressed Log Processor 
>> (CLP) yet? 
>>
>> CLP is a tool capable of losslessly compressing text logs and 
>> searching the compressed logs without decompression.
>>
>> https://github.com/y-scope/clp
>>
>> Chris
>>
> -- 
> You received this message because you are subscribed to a topic in the 
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/golang-nuts/XeDIZfTMlX8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to 
> golang-nuts...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/0c57905a-1096-4688-b268-93742c48316fn%40googlegroups.com
>  
> 
> .
>


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/82c4077a-e6ce-4ac6-b944-cd5ef792a394n%40googlegroups.com.

Re: [go-nuts] Re: Go implementation of CLP

[Jason E. Aten]

CLP looks impressive. 

Is there is a C API then it would probably be easy to write Go bindings to 
it.

Since there is some kind of python integration, perhaps a C API is already 
there.
I took a quick look but could only see C++ code; it would have to be an 
actual C (not C++) API for cgo to be usable.

Usually this is not too difficult to add to a C++ project.

On Wednesday, November 9, 2022 at 2:14:03 AM UTC-6 Bharghava Varun Ayada 
wrote:

> I'm exploring CLP for a logging platform and I think either a cgo or 
> native go library might be something that would help us.
>
> On Tuesday, 4 October 2022 at 22:03:34 UTC+5:30 david...@gmail.com wrote:
>
>> It would be awesome to help SeaweedFS, so I'm eager to hear more about 
>> your ideas to try and understand anything we can do.
>>
>> Sorry if some of my questions are naive. At a high level I'm trying to 
>> better understand two topics:
>> 1. the use cases (or features) we/CLP can help with
>> 2. the technical aspects/limitations that make a pure-go implementation 
>> necessary/beneficial
>>
>> I saw that SeaweedFS currently can automatically compresses certain file 
>> types with Gzip (very cool btw).
>> I can imagine that augmenting this feature with CLP for log file types 
>> could be beneficial.
>> Are these the log files you're referring to? Does SeaweedFS itself also 
>> produce log files that can benefit from CLP? Perhaps CLP could help with 
>> both cases?
>>
>> As I imagine packaging the current form of CLP with SeaweedFS is awkward, 
>> is packaging a main reason for a pure-go implementation?
>> Would a library with programmatic API be favourable for integration 
>> rather than calling out to another tool/executable (as required in CLP's 
>> current form)?
>> Would a go package using cgo (to reuse some existing cpp code) be 
>> desirable or is there a requirement for strictly go code?
>>
>> -david
>>
>> On Sunday, 2 October 2022 at 22:36:38 UTC-4 ChrisLu wrote:
>>
>>> Thanks! CLP is great! 
>>>
>>> I am working on a distributed file system, SeaweedFS, 
>>> https://github.com/seaweedfs/seaweedfs
>>> I am interested in a pure-go implementation to store the log files more 
>>> efficiently.
>>>
>>> Chris
>>>
>>> On Sun, Oct 2, 2022 at 6:45 PM david lion  wrote:
>>>
 Hi Chris,

 I'm one of the CLP developers. We'd be interested in hearing your use 
 case and any features you'd like implemented in CLP.
 As for a Go implementation, are you asking about bindings to directly 
 call CLP functionality from a Go program or are you interested in a 
 complete re-write in Go?

 -david

 On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:

> Seems there are no Go implementation for Compressed Log Processor 
> (CLP) yet? 
>
> CLP is a tool capable of losslessly compressing text logs and 
> searching the compressed logs without decompression.
>
> https://github.com/y-scope/clp
>
> Chris
>
 -- 
 You received this message because you are subscribed to a topic in the 
 Google Groups "golang-nuts" group.
 To unsubscribe from this topic, visit 
 https://groups.google.com/d/topic/golang-nuts/XeDIZfTMlX8/unsubscribe.
 To unsubscribe from this group and all its topics, send an email to 
 golang-nuts...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/golang-nuts/0c57905a-1096-4688-b268-93742c48316fn%40googlegroups.com
  
 
 .

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ce62720d-b67e-4579-a384-7ea821ac1b68n%40googlegroups.com.

Re: [go-nuts] Re: Go implementation of CLP

[Bharghava Varun Ayada]

I'm exploring CLP for a logging platform and I think either a cgo or native 
go library might be something that would help us.

On Tuesday, 4 October 2022 at 22:03:34 UTC+5:30 david...@gmail.com wrote:

> It would be awesome to help SeaweedFS, so I'm eager to hear more about 
> your ideas to try and understand anything we can do.
>
> Sorry if some of my questions are naive. At a high level I'm trying to 
> better understand two topics:
> 1. the use cases (or features) we/CLP can help with
> 2. the technical aspects/limitations that make a pure-go implementation 
> necessary/beneficial
>
> I saw that SeaweedFS currently can automatically compresses certain file 
> types with Gzip (very cool btw).
> I can imagine that augmenting this feature with CLP for log file types 
> could be beneficial.
> Are these the log files you're referring to? Does SeaweedFS itself also 
> produce log files that can benefit from CLP? Perhaps CLP could help with 
> both cases?
>
> As I imagine packaging the current form of CLP with SeaweedFS is awkward, 
> is packaging a main reason for a pure-go implementation?
> Would a library with programmatic API be favourable for integration rather 
> than calling out to another tool/executable (as required in CLP's current 
> form)?
> Would a go package using cgo (to reuse some existing cpp code) be 
> desirable or is there a requirement for strictly go code?
>
> -david
>
> On Sunday, 2 October 2022 at 22:36:38 UTC-4 ChrisLu wrote:
>
>> Thanks! CLP is great! 
>>
>> I am working on a distributed file system, SeaweedFS, 
>> https://github.com/seaweedfs/seaweedfs
>> I am interested in a pure-go implementation to store the log files more 
>> efficiently.
>>
>> Chris
>>
>> On Sun, Oct 2, 2022 at 6:45 PM david lion  wrote:
>>
>>> Hi Chris,
>>>
>>> I'm one of the CLP developers. We'd be interested in hearing your use 
>>> case and any features you'd like implemented in CLP.
>>> As for a Go implementation, are you asking about bindings to directly 
>>> call CLP functionality from a Go program or are you interested in a 
>>> complete re-write in Go?
>>>
>>> -david
>>>
>>> On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:
>>>
 Seems there are no Go implementation for Compressed Log Processor (CLP) 
 yet? 

 CLP is a tool capable of losslessly compressing text logs and searching 
 the compressed logs without decompression.

 https://github.com/y-scope/clp

 Chris

>>> -- 
>>> You received this message because you are subscribed to a topic in the 
>>> Google Groups "golang-nuts" group.
>>> To unsubscribe from this topic, visit 
>>> https://groups.google.com/d/topic/golang-nuts/XeDIZfTMlX8/unsubscribe.
>>> To unsubscribe from this group and all its topics, send an email to 
>>> golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/0c57905a-1096-4688-b268-93742c48316fn%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/63f2d52c-fc2b-44ef-a27e-e5d695626f74n%40googlegroups.com.

Re: [go-nuts] Re: Go implementation of CLP

[david lion]

It would be awesome to help SeaweedFS, so I'm eager to hear more about your 
ideas to try and understand anything we can do.

Sorry if some of my questions are naive. At a high level I'm trying to 
better understand two topics:
1. the use cases (or features) we/CLP can help with
2. the technical aspects/limitations that make a pure-go implementation 
necessary/beneficial

I saw that SeaweedFS currently can automatically compresses certain file 
types with Gzip (very cool btw).
I can imagine that augmenting this feature with CLP for log file types 
could be beneficial.
Are these the log files you're referring to? Does SeaweedFS itself also 
produce log files that can benefit from CLP? Perhaps CLP could help with 
both cases?

As I imagine packaging the current form of CLP with SeaweedFS is awkward, 
is packaging a main reason for a pure-go implementation?
Would a library with programmatic API be favourable for integration rather 
than calling out to another tool/executable (as required in CLP's current 
form)?
Would a go package using cgo (to reuse some existing cpp code) be desirable 
or is there a requirement for strictly go code?

-david

On Sunday, 2 October 2022 at 22:36:38 UTC-4 ChrisLu wrote:

> Thanks! CLP is great! 
>
> I am working on a distributed file system, SeaweedFS, 
> https://github.com/seaweedfs/seaweedfs
> I am interested in a pure-go implementation to store the log files more 
> efficiently.
>
> Chris
>
> On Sun, Oct 2, 2022 at 6:45 PM david lion  wrote:
>
>> Hi Chris,
>>
>> I'm one of the CLP developers. We'd be interested in hearing your use 
>> case and any features you'd like implemented in CLP.
>> As for a Go implementation, are you asking about bindings to directly 
>> call CLP functionality from a Go program or are you interested in a 
>> complete re-write in Go?
>>
>> -david
>>
>> On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:
>>
>>> Seems there are no Go implementation for Compressed Log Processor (CLP) 
>>> yet? 
>>>
>>> CLP is a tool capable of losslessly compressing text logs and searching 
>>> the compressed logs without decompression.
>>>
>>> https://github.com/y-scope/clp
>>>
>>> Chris
>>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/golang-nuts/XeDIZfTMlX8/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/0c57905a-1096-4688-b268-93742c48316fn%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/300d1206-04be-4d82-83b9-3ced848a39dbn%40googlegroups.com.

Re: [go-nuts] Re: Go implementation of CLP

[Chris Lu]

Thanks! CLP is great!

I am working on a distributed file system, SeaweedFS,
https://github.com/seaweedfs/seaweedfs
I am interested in a pure-go implementation to store the log files more
efficiently.

Chris

On Sun, Oct 2, 2022 at 6:45 PM david lion  wrote:

> Hi Chris,
>
> I'm one of the CLP developers. We'd be interested in hearing your use case
> and any features you'd like implemented in CLP.
> As for a Go implementation, are you asking about bindings to directly call
> CLP functionality from a Go program or are you interested in a complete
> re-write in Go?
>
> -david
>
> On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:
>
>> Seems there are no Go implementation for Compressed Log Processor (CLP)
>> yet?
>>
>> CLP is a tool capable of losslessly compressing text logs and searching
>> the compressed logs without decompression.
>>
>> https://github.com/y-scope/clp
>>
>> Chris
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/XeDIZfTMlX8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/0c57905a-1096-4688-b268-93742c48316fn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CACJc9-0gggBXKL0at9k7zKi-KOkXrp5psjQ-W-c-9gdr5QiBAg%40mail.gmail.com.

[go-nuts] Re: Go implementation of CLP

[david lion]

Hi Chris,

I'm one of the CLP developers. We'd be interested in hearing your use case 
and any features you'd like implemented in CLP.
As for a Go implementation, are you asking about bindings to directly call 
CLP functionality from a Go program or are you interested in a complete 
re-write in Go?

-david

On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:

> Seems there are no Go implementation for Compressed Log Processor (CLP) 
> yet? 
>
> CLP is a tool capable of losslessly compressing text logs and searching 
> the compressed logs without decompression.
>
> https://github.com/y-scope/clp
>
> Chris
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0c57905a-1096-4688-b268-93742c48316fn%40googlegroups.com.