I have run into the same problem.
I included the files in the setup.py as in the example but I am not able to
load the data with pkgutil.get_data().
Can someone please post a sample code for that?
I am deploying the spiders on scrapyd and I want to read all the .json
files in conf/
Thanks.
On Thursday, March 14, 2013 at 6:52:46 PM UTC+2, Pablo Hoffman wrote:
>
> This is a snippet I've used to make deploy work with static files:
>
> from setuptools import setup, find_packages
>
> setup(
> name='project',
> version='1.0',
> packages=find_packages(),
> package_data={
> 'myproject': ['conf/*.json', 'conf/cities.txt'],
> },
> entry_points={
> 'scrapy': ['settings = myproject.settings']
> },
> zip_safe=False,
> )
>
>
> The conf/ dir is inside the myproject folder.
>
> And then you use myproject.__file__ to access the folder, on your code.
>
> Pablo.
>
>
>
> On Sat, Feb 16, 2013 at 10:55 PM, Thimo Brinkmann <
> thimo.b...@googlemail.com <javascript:>> wrote:
>
>> Hey guys,
>>
>> I am having a really hard time deploying to scrapyd because of the static
>> configuration files (lets call them dictionary files) that need to be used).
>> Played around with MANIFEST.in, pkgutil.get_data, StringIO and whatever I
>> could find for now more than 4 hours, but really can't get it work. Has
>> anyone a working example?
>>
>> I want to load two static JSON files from scrapyd, but it never seemed to
>> find the files, whatever referencing method I used. Normally I had the open
>> method followed by the filename and I put the filename in the eggs root as
>> well as the projects egg folder, but in no case the files were found. If
>> anyone knows how to do this with a full example, I would be very helpful.
>>
>>
>> Kind regards,
>> Thimo
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "scrapy-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to scrapy-users...@googlegroups.com <javascript:>.
>> To post to this group, send email to scrapy...@googlegroups.com
>> <javascript:>.
>> Visit this group at http://groups.google.com/group/scrapy-users?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to scrapy-users+unsubscr...@googlegroups.com.
To post to this group, send email to scrapy-users@googlegroups.com.
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
