Re: [scikit-learn] is Sci_kiet-Learn the right choice for my project

[Mike Oliver]

Bill,

Granted, you are correct the examples I gave are simple enough for a set of 
rules to process.  The problem as I see it is that with the plethora of 
potential relationships, the rules may not be adequate.  Inconsistent data is 
not transactional for example I can store data on a pair of objects and they 
pass all the rules.  Then later another relationship is stored, and that passes 
all the rules…yet just because A is consistent with B and B is consistent with 
C does NOT mean A is consistent with C.  Anomaly detection can be simple or it 
can be complex.  But if 10,000 relationships have been stored and one changes I 
want an algorithm to emerge that we can see and turn into an action to fix it.

We are also hoping that things may emerge that we did not anticipate.  That may 
require deep learning sub-symbolic neural networks, but that is yet to be 
determined.

And yes, with literally thousands of records flowing through the system, the 
delay in processing each record against every other record in a read before 
write model is not going to perform well.  If we can use machine learning we 
can take that evaluation and even corrections out of that processing flow.

Thanks,

If scikit-learn is not a good fit for my goals, let me know.  If you know a 
better fit, please let me know as well.

Mike

From: Bill Ross 
Sent: Sunday, October 9, 2022 12:46 AM
To: Scikit-learn mailing list 
Cc: scikit-learn ; 
Mike Oliver 
Subject: Re: [scikit-learn] is Sci_kiet-Learn the right choice for my project

[Warning]CAUTION: This email originated from outside of the organization. Do 
not click on links or open attachments unless you recognize the sender and know 
that the contents are safe.

>   My hope is that with machine learning we can detect when an object is 
> missing, or configured in error, or duplicates.

These look like simple correctness issues that I'd address with programming.
Why do you want to use a learned approach? Do you think it will be faster to 
develop, or have a faster runtime?

Bill

--

Phobrain.com



On 2022-10-08 01:57, Mike Oliver wrote:
Dear Sirs,

I am evaluating SciKit-Learn for a new project.  I am hoping to find a AI 
Machine Learning package that can take a large dataset of objects that have 
various object types and attributes.  These objects are typically related to 
other objects, such as a server to a Wifi device, or two network routers to 
each other, etc.  When these objects are setup data is gathered about where 
they are located, what settings there are, the device type, etc.

With large organizations there can be thousands of these objects and tens of 
thousands of relationships, descriptions, settings, etc.  My hope is that with 
machine learning we can detect when an object is missing, or configured in 
error, or duplicates.

The question is, will SciKit-Learn help with this problem? I understand that we 
will have to train it to identify what to look for and then act on what was 
found and predicted to be the solution algorithm. Or instructions.

Thanks for your help,

Great looking product and already have the tutorial up and running and have 
installed it in my Django platform.

Mike

___
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn
___
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn

Re: [scikit-learn] is Sci_kiet-Learn the right choice for my project

[Bill Ross]

>   My hope is that with machine learning we can detect when an object is 
> missing, or configured in error, or duplicates. 

These look like simple correctness issues that I'd address with
programming.

Why do you want to use a learned approach? Do you think it will be
faster to develop, or have a faster runtime?  

Bill 

--

Phobrain.com 

On 2022-10-08 01:57, Mike Oliver wrote:

> Dear Sirs, 
> 
> I am evaluating SciKit-Learn for a new project.  I am hoping to find a AI 
> Machine Learning package that can take a large dataset of objects that have 
> various object types and attributes.  These objects are typically related to 
> other objects, such as a server to a Wifi device, or two network routers to 
> each other, etc.  When these objects are setup data is gathered about where 
> they are located, what settings there are, the device type, etc. 
> 
> With large organizations there can be thousands of these objects and tens of 
> thousands of relationships, descriptions, settings, etc.  My hope is that 
> with machine learning we can detect when an object is missing, or configured 
> in error, or duplicates. 
> 
> The question is, will SciKit-Learn help with this problem? I understand that 
> we will have to train it to identify what to look for and then act on what 
> was found and predicted to be the solution algorithm. Or instructions. 
> 
> Thanks for your help, 
> 
> Great looking product and already have the tutorial up and running and have 
> installed it in my Django platform. 
> 
> Mike 
> ___
> scikit-learn mailing list
> scikit-learn@python.org
> https://mail.python.org/mailman/listinfo/scikit-learn___
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn

Re: [scikit-learn] is Sci_kiet-Learn the right choice for my project

[Brown J.B. via scikit-learn]

Dear Mike,

Just my two cents about your inquiry, where I strictly a user of
scikit-learn for many years.

- From your description of application context, I would say that
scikit-learn is perfectly fine. However, I would suggest the awareness that
a monolithic model incorporating all data (as is the image TV wrongfully
projects) is not a valid strategy. Stratifying data into contextually
correct subgroups and then running scikit-learn, for example to estimate
during development the extent of predictability, will be helpful.
- Duplicate checking should be easy to use using standard python objects
(set or list counting), once the context derives how the objects are
vectorized/featurized. I don't see a need to force scikit-learn for that
context.
- Missing data could be implemented by context-specific object classes that
you design, which could contain something like a __bool__()  method that
could tell if you if the object has all of the required data populated and
configured.
- Detection of errors in configuration could be either explicitly driven by
logic (of the context, again something to return a bool that an object is
configured correctly), or potentially could be statistically derived as
outliers from the given background data distribution, in which then
scikit-learn could be of help. If there are too many variates (thousands or
tens of thousands) in your data that prohibit explicit logic, then
scikit-learn's Random Forest algorithms might be perfectly fine and provide
verification through visualization of Decision Tree rules.

Hope this helps,
J.B. Brown

2022年10月8日(土) 10:59 Mike Oliver :

> Dear Sirs,
>
>
>
> I am evaluating SciKit-Learn for a new project.  I am hoping to find a AI
> Machine Learning package that can take a large dataset of objects that have
> various object types and attributes.  These objects are typically related
> to other objects, such as a server to a Wifi device, or two network routers
> to each other, etc.  When these objects are setup data is gathered about
> where they are located, what settings there are, the device type, etc.
>
>
>
> With large organizations there can be thousands of these objects and tens
> of thousands of relationships, descriptions, settings, etc.  My hope is
> that with machine learning we can detect when an object is missing, or
> configured in error, or duplicates.
>
>
>
> The question is, will SciKit-Learn help with this problem? I understand
> that we will have to train it to identify what to look for and then act on
> what was found and predicted to be the solution algorithm. Or instructions.
>
>
>
> Thanks for your help,
>
>
>
> Great looking product and already have the tutorial up and running and
> have installed it in my Django platform.
>
>
>
> Mike
> ___
> scikit-learn mailing list
> scikit-learn@python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
___
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn

Re: [scikit-learn] is Sci_kiet-Learn the right choice for my project

[Bob Helmbold via scikit-learn]

Mebbe Tensor Flow would be a better fit to your needs.
On 10/08/2022 1:58:03 AM, Mike Oliver  wrote:
Dear Sirs,
 
I am evaluating SciKit-Learn for a new project.  I am hoping to find a AI 
Machine Learning package that can take a large dataset of objects that have 
various object types and attributes.  These objects are typically related to 
other objects, such as a server to a Wifi device, or two network routers to 
each other, etc.  When these objects are setup data is gathered about where 
they are located, what settings there are, the device type, etc.
 
With large organizations there can be thousands of these objects and tens of 
thousands of relationships, descriptions, settings, etc.  My hope is that with 
machine learning we can detect when an object is missing, or configured in 
error, or duplicates.
 
The question is, will SciKit-Learn help with this problem? I understand that we 
will have to train it to identify what to look for and then act on what was 
found and predicted to be the solution algorithm. Or instructions.
 
Thanks for your help,
 
Great looking product and already have the tutorial up and running and have 
installed it in my Django platform.
 
Mike___
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn

[scikit-learn] is Sci_kiet-Learn the right choice for my project

[Mike Oliver]

Dear Sirs,

I am evaluating SciKit-Learn for a new project.  I am hoping to find a AI 
Machine Learning package that can take a large dataset of objects that have 
various object types and attributes.  These objects are typically related to 
other objects, such as a server to a Wifi device, or two network routers to 
each other, etc.  When these objects are setup data is gathered about where 
they are located, what settings there are, the device type, etc.

With large organizations there can be thousands of these objects and tens of 
thousands of relationships, descriptions, settings, etc.  My hope is that with 
machine learning we can detect when an object is missing, or configured in 
error, or duplicates.

The question is, will SciKit-Learn help with this problem? I understand that we 
will have to train it to identify what to look for and then act on what was 
found and predicted to be the solution algorithm. Or instructions.

Thanks for your help,

Great looking product and already have the tutorial up and running and have 
installed it in my Django platform.

Mike
___
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn