On Friday, 12 June 2020 at 18:22:28 UTC, Selim wrote:
I wrote a small Yahoo finance scraper and wanted to share with
the community. I have been using D for a while and I think
contributing something to the community is good. There is an
example main script and a unit test. Those should get you
going. It currently saves the scraped data as a json file under
the executable's folder. I might add a public method to access
individual data columns inside json in the following days too.
All mistakes are my own and I appreciate any feedback.
https://github.com/SelimOzel/YahooMinerD
Best,
Selim
Thanks! There was a period there where you couldn't use the yahoo
API, glad to see that people can use it again.
I haven't run it myself yet, but I have a few comments for
potential changes and enhancements.
Why do you use a class for YahooMinerD (also the name
YahooFinanceD might get more people to use)? It doesn't look like
you are using any inheritance. I don't see any reason not to
change to a struct and avoid new.
It looks like you have a lot of writeln statements. While these
could be helpful, they will also prevent those functions from
ever being @nogc. You could use an approach like below and give
the user the opportunity to avoid the writelns and allow for
attribute inference elsewhere.
@nogc void fooImpl(bool val)()
if (val)
{
}
void fooImpl(bool val)()
if (!val)
{
import std.stdio: writeln;
writeln("here");
}
void foo(bool val = false)()
{
fooImpl!val;
}
@nogc void main() {
foo!true;
}
It looks like the primary way to get the data is from the
WriteToJson, correct? What if you want to use the data without
writing the JSON to file? For instance, I want to get the data
and put it in my own database, or I just want to get the data, do
some calculations, and then not save it. There should be a way to
get the data out of there without writing to file. std.json can
be used for parsing the JSON and there are other libraries out
there.
The WriteToJSON method should also allow writing events or prices
without needing to write both.
You might consider adding the ability to control the frequency
(instead of just daily).
