Send Beginners mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://www.haskell.org/mailman/listinfo/beginners
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Beginners digest..."
Today's Topics:
1. Re: Nonsensical behavior with http-conduit, httpLbs
(Christopher Allen)
2. Re: Nonsensical behavior with http-conduit, httpLbs
(Christopher Allen)
3. Re: Nonsensical behavior with http-conduit, httpLbs
(Michael Snoyman)
----------------------------------------------------------------------
Message: 1
Date: Thu, 10 Apr 2014 01:25:02 -0500
From: Christopher Allen <[email protected]>
To: The Haskell-Beginners Mailing List - Discussion of primarily
beginner-level topics related to Haskell <[email protected]>
Subject: Re: [Haskell-beginners] Nonsensical behavior with
http-conduit, httpLbs
Message-ID:
<cadnndoryky8avcz0m5fvhl6byotu-0wkwdmq1dnszthv1h4...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
It's a step removed from that, and `seq`'ing insertData isn't fixing it,
only a threadDelay is.
The operations insertData is performing are:
delete index, create index, add document to index
the operation queryTweet is performing is:
search index
The behavior seems to indicate "search index" is happening before "add
document to index" and after "delete index", as the document will not exist
when the query is performed.
Only adding a threadDelay as reliably worked. Even `seq`'ing insertData
doesn't work.
Are the requests somehow being performed concurrently?
On Thu, Apr 10, 2014 at 1:15 AM, Michael Snoyman <[email protected]>wrote:
> I can't say I fully understand your code, but it seems like it's doing the
> following:
>
> 1. Send request to server to delete data.
> 2. Send request to server to add data.
> 3. Request data from server.
>
> You're saying that with a long enough delay between steps 2 and 3, then
> (3) works, otherwise it fails. It sounds to me like there's some kind of a
> race condition. I don't know anything about createExampleIndex, but are you
> certain that it only returns after the data is completely ready for
> querying?
>
>
> On Thu, Apr 10, 2014 at 8:43 AM, Christopher Allen <[email protected]>wrote:
>
>> to add to my question adding a delay made it work:
>>
>> main :: IO ()
>> main = do
>> _ <- insertData
>> threadDelay 1000000
>> myTweet <- queryTweet
>> print myTweet
>>
>>
>>
>> On Thu, Apr 10, 2014 at 12:41 AM, Christopher Allen
>> <[email protected]>wrote:
>>
>>> Repro'd on GHC 7.8.1 and 7.6.3
>>>
>>> Original code is: https://github.com/bitemyapp/bloodhound/
>>>
>>> The code only works if I manually (out of band) run insertData (in a
>>> REPL) and then run main with the insertData invocation stripped out. If I
>>> run main with the insertData invocation included, it throws an exception
>>> (head) because the search results are empty.
>>>
>>> The behavior is as if queryTweet was executing after insertData deleted
>>> the index, but before it inserted the new data.
>>>
>>> The following is a stripped down example:
>>>
>>> insertData :: IO ()
>>> insertData = do
>>> let encoded = encode exampleTweet
>>> _ <- deleteExampleIndex
>>> created <- createExampleIndex
>>> docCreated <- indexDocument (Server "http://localhost:9200";)
>>> "twitter" "tweet" exampleTweet "1"
>>> print "test"
>>> return ()
>>> el
>>> queryTweet :: IO (Either String Tweet)
>>> queryTweet = do
>>> let queryFilter = BoolFilter (MustMatch (Term "user" "bitemyapp")
>>> False)
>>> <||> IdentityFilter
>>> let search = Search Nothing (Just queryFilter)
>>> reply <- searchByIndex testServer "twitter" search
>>> let result = eitherDecode (responseBody reply) :: Either String
>>> (SearchResult Tweet)
>>> let myTweet = fmap (hitSource . head . hits . searchHits) result
>>> return myTweet
>>>
>>> main :: IO ()
>>> main = do
>>> _ <- insertData
>>> myTweet <- queryTweet
>>> print myTweet
>>>
>>> further up the call chain, the http call is getting dispatched with
>>> http-conduit's gear, the result returned with the expression:
>>>
>>> withManager $ httpLbs req
>>>
>>> Included this in case it affects the semantics.
>>>
>>> Can anybody help? This has stumped me for a couple hours and I couldn't
>>> get anything clueful on IRC.
>>>
>>> Thanks to any that help.
>>>
>>> --- Chris
>>>
>>>
>>
>> _______________________________________________
>> Beginners mailing list
>> [email protected]
>> http://www.haskell.org/mailman/listinfo/beginners
>>
>>
>
> _______________________________________________
> Beginners mailing list
> [email protected]
> http://www.haskell.org/mailman/listinfo/beginners
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.haskell.org/pipermail/beginners/attachments/20140410/dac5c7be/attachment-0001.html>
------------------------------
Message: 2
Date: Thu, 10 Apr 2014 01:25:38 -0500
From: Christopher Allen <[email protected]>
To: The Haskell-Beginners Mailing List - Discussion of primarily
beginner-level topics related to Haskell <[email protected]>
Subject: Re: [Haskell-beginners] Nonsensical behavior with
http-conduit, httpLbs
Message-ID:
<cadnndop+tniaa2qcb3_uudddc74uowsvwsbn1dxjdzfqu2w...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
the function actually running the requests is:
dispatch :: String -> Method -> Maybe L.ByteString
-> IO Reply
dispatch url method body = do
initReq <- parseUrl url
let reqBody = RequestBodyLBS $ fromMaybe emptyBody body
let req = initReq { method = method
, requestBody = reqBody
, checkStatus = \_ _ _ -> Nothing}
withManager $ httpLbs req
rest of the code is here: https://github.com/bitemyapp/bloodhound/
On Thu, Apr 10, 2014 at 1:25 AM, Christopher Allen <[email protected]>wrote:
> It's a step removed from that, and `seq`'ing insertData isn't fixing it,
> only a threadDelay is.
>
> The operations insertData is performing are:
> delete index, create index, add document to index
>
> the operation queryTweet is performing is:
> search index
>
> The behavior seems to indicate "search index" is happening before "add
> document to index" and after "delete index", as the document will not exist
> when the query is performed.
>
> Only adding a threadDelay as reliably worked. Even `seq`'ing insertData
> doesn't work.
>
> Are the requests somehow being performed concurrently?
>
>
>
> On Thu, Apr 10, 2014 at 1:15 AM, Michael Snoyman <[email protected]>wrote:
>
>> I can't say I fully understand your code, but it seems like it's doing
>> the following:
>>
>> 1. Send request to server to delete data.
>> 2. Send request to server to add data.
>> 3. Request data from server.
>>
>> You're saying that with a long enough delay between steps 2 and 3, then
>> (3) works, otherwise it fails. It sounds to me like there's some kind of a
>> race condition. I don't know anything about createExampleIndex, but are you
>> certain that it only returns after the data is completely ready for
>> querying?
>>
>>
>> On Thu, Apr 10, 2014 at 8:43 AM, Christopher Allen <[email protected]>wrote:
>>
>>> to add to my question adding a delay made it work:
>>>
>>> main :: IO ()
>>> main = do
>>> _ <- insertData
>>> threadDelay 1000000
>>> myTweet <- queryTweet
>>> print myTweet
>>>
>>>
>>>
>>> On Thu, Apr 10, 2014 at 12:41 AM, Christopher Allen
>>> <[email protected]>wrote:
>>>
>>>> Repro'd on GHC 7.8.1 and 7.6.3
>>>>
>>>> Original code is: https://github.com/bitemyapp/bloodhound/
>>>>
>>>> The code only works if I manually (out of band) run insertData (in a
>>>> REPL) and then run main with the insertData invocation stripped out. If I
>>>> run main with the insertData invocation included, it throws an exception
>>>> (head) because the search results are empty.
>>>>
>>>> The behavior is as if queryTweet was executing after insertData deleted
>>>> the index, but before it inserted the new data.
>>>>
>>>> The following is a stripped down example:
>>>>
>>>> insertData :: IO ()
>>>> insertData = do
>>>> let encoded = encode exampleTweet
>>>> _ <- deleteExampleIndex
>>>> created <- createExampleIndex
>>>> docCreated <- indexDocument (Server "http://localhost:9200";)
>>>> "twitter" "tweet" exampleTweet "1"
>>>> print "test"
>>>> return ()
>>>> el
>>>> queryTweet :: IO (Either String Tweet)
>>>> queryTweet = do
>>>> let queryFilter = BoolFilter (MustMatch (Term "user" "bitemyapp")
>>>> False)
>>>> <||> IdentityFilter
>>>> let search = Search Nothing (Just queryFilter)
>>>> reply <- searchByIndex testServer "twitter" search
>>>> let result = eitherDecode (responseBody reply) :: Either String
>>>> (SearchResult Tweet)
>>>> let myTweet = fmap (hitSource . head . hits . searchHits) result
>>>> return myTweet
>>>>
>>>> main :: IO ()
>>>> main = do
>>>> _ <- insertData
>>>> myTweet <- queryTweet
>>>> print myTweet
>>>>
>>>> further up the call chain, the http call is getting dispatched with
>>>> http-conduit's gear, the result returned with the expression:
>>>>
>>>> withManager $ httpLbs req
>>>>
>>>> Included this in case it affects the semantics.
>>>>
>>>> Can anybody help? This has stumped me for a couple hours and I couldn't
>>>> get anything clueful on IRC.
>>>>
>>>> Thanks to any that help.
>>>>
>>>> --- Chris
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Beginners mailing list
>>> [email protected]
>>> http://www.haskell.org/mailman/listinfo/beginners
>>>
>>>
>>
>> _______________________________________________
>> Beginners mailing list
>> [email protected]
>> http://www.haskell.org/mailman/listinfo/beginners
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.haskell.org/pipermail/beginners/attachments/20140410/7829ab37/attachment-0001.html>
------------------------------
Message: 3
Date: Thu, 10 Apr 2014 09:28:32 +0300
From: Michael Snoyman <[email protected]>
To: The Haskell-Beginners Mailing List - Discussion of primarily
beginner-level topics related to Haskell <[email protected]>
Subject: Re: [Haskell-beginners] Nonsensical behavior with
http-conduit, httpLbs
Message-ID:
<CAKA2JgJLMGajXC_sztga+NmcHq=6j0pabkp8vwgp_gmnraq...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
http-conduit does not perform any kind of automatic concurrency. httpLbs
will only return after it has fully read the response body from the server.
So unless there's a bug somewhere in http-conduit, it seems that the server
has not fully updated its index by the time it gives back a response.
On Thu, Apr 10, 2014 at 9:25 AM, Christopher Allen <[email protected]>wrote:
> the function actually running the requests is:
>
> dispatch :: String -> Method -> Maybe L.ByteString
> -> IO Reply
> dispatch url method body = do
> initReq <- parseUrl url
> let reqBody = RequestBodyLBS $ fromMaybe emptyBody body
> let req = initReq { method = method
> , requestBody = reqBody
> , checkStatus = \_ _ _ -> Nothing}
> withManager $ httpLbs req
>
> rest of the code is here: https://github.com/bitemyapp/bloodhound/
>
>
> On Thu, Apr 10, 2014 at 1:25 AM, Christopher Allen <[email protected]>wrote:
>
>> It's a step removed from that, and `seq`'ing insertData isn't fixing it,
>> only a threadDelay is.
>>
>> The operations insertData is performing are:
>> delete index, create index, add document to index
>>
>> the operation queryTweet is performing is:
>> search index
>>
>> The behavior seems to indicate "search index" is happening before "add
>> document to index" and after "delete index", as the document will not exist
>> when the query is performed.
>>
>> Only adding a threadDelay as reliably worked. Even `seq`'ing insertData
>> doesn't work.
>>
>> Are the requests somehow being performed concurrently?
>>
>>
>>
>> On Thu, Apr 10, 2014 at 1:15 AM, Michael Snoyman <[email protected]>wrote:
>>
>>> I can't say I fully understand your code, but it seems like it's doing
>>> the following:
>>>
>>> 1. Send request to server to delete data.
>>> 2. Send request to server to add data.
>>> 3. Request data from server.
>>>
>>> You're saying that with a long enough delay between steps 2 and 3, then
>>> (3) works, otherwise it fails. It sounds to me like there's some kind of a
>>> race condition. I don't know anything about createExampleIndex, but are you
>>> certain that it only returns after the data is completely ready for
>>> querying?
>>>
>>>
>>> On Thu, Apr 10, 2014 at 8:43 AM, Christopher Allen
>>> <[email protected]>wrote:
>>>
>>>> to add to my question adding a delay made it work:
>>>>
>>>> main :: IO ()
>>>> main = do
>>>> _ <- insertData
>>>> threadDelay 1000000
>>>> myTweet <- queryTweet
>>>> print myTweet
>>>>
>>>>
>>>>
>>>> On Thu, Apr 10, 2014 at 12:41 AM, Christopher Allen
>>>> <[email protected]>wrote:
>>>>
>>>>> Repro'd on GHC 7.8.1 and 7.6.3
>>>>>
>>>>> Original code is: https://github.com/bitemyapp/bloodhound/
>>>>>
>>>>> The code only works if I manually (out of band) run insertData (in a
>>>>> REPL) and then run main with the insertData invocation stripped out. If I
>>>>> run main with the insertData invocation included, it throws an exception
>>>>> (head) because the search results are empty.
>>>>>
>>>>> The behavior is as if queryTweet was executing after insertData
>>>>> deleted the index, but before it inserted the new data.
>>>>>
>>>>> The following is a stripped down example:
>>>>>
>>>>> insertData :: IO ()
>>>>> insertData = do
>>>>> let encoded = encode exampleTweet
>>>>> _ <- deleteExampleIndex
>>>>> created <- createExampleIndex
>>>>> docCreated <- indexDocument (Server "http://localhost:9200";)
>>>>> "twitter" "tweet" exampleTweet "1"
>>>>> print "test"
>>>>> return ()
>>>>> el
>>>>> queryTweet :: IO (Either String Tweet)
>>>>> queryTweet = do
>>>>> let queryFilter = BoolFilter (MustMatch (Term "user" "bitemyapp")
>>>>> False)
>>>>> <||> IdentityFilter
>>>>> let search = Search Nothing (Just queryFilter)
>>>>> reply <- searchByIndex testServer "twitter" search
>>>>> let result = eitherDecode (responseBody reply) :: Either String
>>>>> (SearchResult Tweet)
>>>>> let myTweet = fmap (hitSource . head . hits . searchHits) result
>>>>> return myTweet
>>>>>
>>>>> main :: IO ()
>>>>> main = do
>>>>> _ <- insertData
>>>>> myTweet <- queryTweet
>>>>> print myTweet
>>>>>
>>>>> further up the call chain, the http call is getting dispatched with
>>>>> http-conduit's gear, the result returned with the expression:
>>>>>
>>>>> withManager $ httpLbs req
>>>>>
>>>>> Included this in case it affects the semantics.
>>>>>
>>>>> Can anybody help? This has stumped me for a couple hours and I
>>>>> couldn't get anything clueful on IRC.
>>>>>
>>>>> Thanks to any that help.
>>>>>
>>>>> --- Chris
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Beginners mailing list
>>>> [email protected]
>>>> http://www.haskell.org/mailman/listinfo/beginners
>>>>
>>>>
>>>
>>> _______________________________________________
>>> Beginners mailing list
>>> [email protected]
>>> http://www.haskell.org/mailman/listinfo/beginners
>>>
>>>
>>
>
> _______________________________________________
> Beginners mailing list
> [email protected]
> http://www.haskell.org/mailman/listinfo/beginners
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.haskell.org/pipermail/beginners/attachments/20140410/7765f946/attachment.html>
------------------------------
Subject: Digest Footer
_______________________________________________
Beginners mailing list
[email protected]
http://www.haskell.org/mailman/listinfo/beginners
------------------------------
End of Beginners Digest, Vol 70, Issue 21
*****************************************
