- Revision
- 152976
- Author
- [email protected]
- Date
- 2013-07-22 09:35:15 -0700 (Mon, 22 Jul 2013)
Log Message
[Curl] Download fails for certain urls.
https://bugs.webkit.org/show_bug.cgi?id=118468
Patch by [email protected] <[email protected]> on 2013-07-22
Reviewed by Brent Fulgham.
Source/WebCore:
The current Curl download implementation has a few shortcomings:
1) Downloading from secure locations fails. We need to provide Curl with the path to a certificate file (.pem file).
2) Cookies are not set in the download request. We need to give Curl the path to the cookie file.
3) When a normal load is converted to a download, some of the headers from the original request is not sent (e.g. Referer, User agent).
* platform/network/curl/CurlDownload.cpp:
(WebCore::CurlDownload::CurlDownload): Initialize custom headers member.
(WebCore::CurlDownload::~CurlDownload): Free custom headers member.
(WebCore::CurlDownload::init): Set certificate and cookie file path.
(WebCore::CurlDownload::closeFile): Check file handle against value for invalid platform handle.
(WebCore::CurlDownload::writeDataToFile): Added utility method to write download data to file.
(WebCore::CurlDownload::addHeaders): Added utility method to add headers to request.
(WebCore::CurlDownload::didReceiveData): Use writeDataToFile utility method.
* platform/network/curl/CurlDownload.h:
Put class in WebCore namespace.
Added method to init download from resource handle, request, and response object.
Added utility method to write download data to file.
Added utility method to add headers to request.
Added custom headers member.
Source/WebKit/win:
Initialize download from provided resource handle, request, and response object.
* WebDownload.h: Use WebCore namespace for Curl download class.
* WebDownloadCurl.cpp:
(WebDownload::init): Initialize download from provided resource handle, request, and response object.
Modified Paths
Diff
Modified: trunk/Source/WebCore/ChangeLog (152975 => 152976)
--- trunk/Source/WebCore/ChangeLog 2013-07-22 16:26:43 UTC (rev 152975)
+++ trunk/Source/WebCore/ChangeLog 2013-07-22 16:35:15 UTC (rev 152976)
@@ -1,3 +1,30 @@
+2013-07-22 [email protected] <[email protected]>
+
+ [Curl] Download fails for certain urls.
+ https://bugs.webkit.org/show_bug.cgi?id=118468
+
+ Reviewed by Brent Fulgham.
+
+ The current Curl download implementation has a few shortcomings:
+ 1) Downloading from secure locations fails. We need to provide Curl with the path to a certificate file (.pem file).
+ 2) Cookies are not set in the download request. We need to give Curl the path to the cookie file.
+ 3) When a normal load is converted to a download, some of the headers from the original request is not sent (e.g. Referer, User agent).
+
+ * platform/network/curl/CurlDownload.cpp:
+ (WebCore::CurlDownload::CurlDownload): Initialize custom headers member.
+ (WebCore::CurlDownload::~CurlDownload): Free custom headers member.
+ (WebCore::CurlDownload::init): Set certificate and cookie file path.
+ (WebCore::CurlDownload::closeFile): Check file handle against value for invalid platform handle.
+ (WebCore::CurlDownload::writeDataToFile): Added utility method to write download data to file.
+ (WebCore::CurlDownload::addHeaders): Added utility method to add headers to request.
+ (WebCore::CurlDownload::didReceiveData): Use writeDataToFile utility method.
+ * platform/network/curl/CurlDownload.h:
+ Put class in WebCore namespace.
+ Added method to init download from resource handle, request, and response object.
+ Added utility method to write download data to file.
+ Added utility method to add headers to request.
+ Added custom headers member.
+
2013-07-22 Eric Carlson <[email protected]>
[iOS] captions sometimes positioned incorrectly after fullscreen state change
Modified: trunk/Source/WebCore/platform/network/curl/CurlDownload.cpp (152975 => 152976)
--- trunk/Source/WebCore/platform/network/curl/CurlDownload.cpp 2013-07-22 16:26:43 UTC (rev 152975)
+++ trunk/Source/WebCore/platform/network/curl/CurlDownload.cpp 2013-07-22 16:35:15 UTC (rev 152976)
@@ -28,6 +28,7 @@
#include <WebCore/HTTPParsers.h>
#include <WebCore/MainThreadTask.h>
+#include <WebCore/ResourceRequest.h>
#include <wtf/MainThread.h>
#include <wtf/text/CString.h>
@@ -37,6 +38,8 @@
template<> struct CrossThreadCopierBase<false, false, CurlDownload*> : public CrossThreadCopierPassThrough<CurlDownload*> {
};
+namespace WebCore {
+
// CurlDownloadManager -------------------------------------------------------------------
CurlDownloadManager::CurlDownloadManager()
@@ -213,6 +216,7 @@
CurlDownload::CurlDownload()
: m_curlHandle(0)
+, m_customHeaders(0)
, m_url(0)
, m_tempHandle(invalidPlatformFileHandle)
, m_deletesFileUponFailure(false)
@@ -227,6 +231,9 @@
if (m_url)
fastFree(m_url);
+ if (m_customHeaders)
+ curl_slist_free_all(m_customHeaders);
+
closeFile();
moveFileToDestination();
}
@@ -253,9 +260,31 @@
curl_easy_setopt(m_curlHandle, CURLOPT_MAXREDIRS, 10);
curl_easy_setopt(m_curlHandle, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
+ const char* certPath = getenv("CURL_CA_BUNDLE_PATH");
+ if (certPath)
+ curl_easy_setopt(m_curlHandle, CURLOPT_CAINFO, certPath);
+
+ const char* cookieJarPath = getenv("CURL_COOKIE_JAR_PATH");
+ if (cookieJarPath)
+ curl_easy_setopt(m_curlHandle, CURLOPT_COOKIEFILE, cookieJarPath);
+
m_listener = listener;
}
+void CurlDownload::init(CurlDownloadListener* listener, ResourceHandle*, const ResourceRequest& request, const ResourceResponse&)
+{
+ if (!listener)
+ return;
+
+ MutexLocker locker(m_mutex);
+
+ KURL url(ParsedURLString, request.url());
+
+ init(listener, url);
+
+ addHeaders(request);
+}
+
bool CurlDownload::start()
{
return m_downloadManager.add(m_curlHandle);
@@ -288,7 +317,7 @@
{
MutexLocker locker(m_mutex);
- if (m_tempHandle) {
+ if (m_tempHandle != invalidPlatformFileHandle) {
WebCore::closeFile(m_tempHandle);
m_tempHandle = invalidPlatformFileHandle;
}
@@ -302,6 +331,43 @@
::MoveFileEx(m_tempPath.charactersWithNullTermination().data(), m_destination.charactersWithNullTermination().data(), MOVEFILE_COPY_ALLOWED | MOVEFILE_REPLACE_EXISTING);
}
+void CurlDownload::writeDataToFile(const char* data, int size)
+{
+ if (m_tempPath.isEmpty())
+ m_tempPath = openTemporaryFile("download", m_tempHandle);
+
+ if (m_tempHandle != invalidPlatformFileHandle)
+ writeToFile(m_tempHandle, data, size);
+}
+
+void CurlDownload::addHeaders(const ResourceRequest& request)
+{
+ if (request.httpHeaderFields().size() > 0) {
+ struct curl_slist* headers = 0;
+
+ HTTPHeaderMap customHeaders = request.httpHeaderFields();
+ HTTPHeaderMap::const_iterator end = customHeaders.end();
+ for (HTTPHeaderMap::const_iterator it = customHeaders.begin(); it != end; ++it) {
+ const String& value = it->value;
+ String headerString(it->key);
+ if (value.isEmpty())
+ // Insert the ; to tell curl that this header has an empty value.
+ headerString.append(";");
+ else {
+ headerString.append(": ");
+ headerString.append(value);
+ }
+ CString headerLatin1 = headerString.latin1();
+ headers = curl_slist_append(headers, headerLatin1.data());
+ }
+
+ if (headers) {
+ curl_easy_setopt(m_curlHandle, CURLOPT_HTTPHEADER, headers);
+ m_customHeaders = headers;
+ }
+ }
+}
+
void CurlDownload::didReceiveHeader(const String& header)
{
MutexLocker locker(m_mutex);
@@ -335,13 +401,7 @@
callOnMainThread<CurlDownload*, CurlDownload*, int, int>(receivedDataCallback, this, size);
- if (m_tempPath.isEmpty())
- m_tempPath = openTemporaryFile("download", m_tempHandle);
-
- if (m_tempHandle != invalidPlatformFileHandle) {
- const char* fileData = static_cast<const char*>(data);
- writeToFile(m_tempHandle, fileData, size);
- }
+ writeDataToFile(static_cast<const char*>(data), size);
}
void CurlDownload::didReceiveResponse()
@@ -425,3 +485,5 @@
if (download)
download->didReceiveResponse();
}
+
+}
Modified: trunk/Source/WebCore/platform/network/curl/CurlDownload.h (152975 => 152976)
--- trunk/Source/WebCore/platform/network/curl/CurlDownload.h 2013-07-22 16:26:43 UTC (rev 152975)
+++ trunk/Source/WebCore/platform/network/curl/CurlDownload.h 2013-07-22 16:35:15 UTC (rev 152976)
@@ -27,6 +27,7 @@
#define CurlDownload_h
#include <WebCore/FileSystem.h>
+#include <WebCore/ResourceHandle.h>
#include <WebCore/ResourceResponse.h>
#if PLATFORM(WIN)
@@ -36,6 +37,8 @@
#include <curl/curl.h>
+namespace WebCore {
+
class CurlDownloadManager {
public:
CurlDownloadManager();
@@ -87,6 +90,8 @@
~CurlDownload();
void init(CurlDownloadListener*, const WebCore::KURL&);
+ void init(CurlDownloadListener*, ResourceHandle*, const ResourceRequest&, const ResourceResponse&);
+
bool start();
bool cancel();
@@ -102,7 +107,10 @@
private:
void closeFile();
void moveFileToDestination();
+ void writeDataToFile(const char* data, int size);
+ void addHeaders(const ResourceRequest&);
+
// Called on download thread.
void didReceiveHeader(const String& header);
void didReceiveData(void* data, int size);
@@ -122,6 +130,7 @@
static void receivedResponseCallback(CurlDownload*);
CURL* m_curlHandle;
+ struct curl_slist* m_customHeaders;
char* m_url;
String m_tempPath;
String m_destination;
@@ -136,4 +145,6 @@
friend class CurlDownloadManager;
};
+}
+
#endif
Modified: trunk/Source/WebKit/win/ChangeLog (152975 => 152976)
--- trunk/Source/WebKit/win/ChangeLog 2013-07-22 16:26:43 UTC (rev 152975)
+++ trunk/Source/WebKit/win/ChangeLog 2013-07-22 16:35:15 UTC (rev 152976)
@@ -1,3 +1,16 @@
+2013-07-22 [email protected] <[email protected]>
+
+ [Curl] Download fails for certain urls.
+ https://bugs.webkit.org/show_bug.cgi?id=118468
+
+ Reviewed by Brent Fulgham.
+
+ Initialize download from provided resource handle, request, and response object.
+
+ * WebDownload.h: Use WebCore namespace for Curl download class.
+ * WebDownloadCurl.cpp:
+ (WebDownload::init): Initialize download from provided resource handle, request, and response object.
+
2013-07-19 [email protected] <[email protected]>
[Curl] Http response code 401 (Authentication required) is not handled.
Modified: trunk/Source/WebKit/win/WebDownload.h (152975 => 152976)
--- trunk/Source/WebKit/win/WebDownload.h 2013-07-22 16:26:43 UTC (rev 152975)
+++ trunk/Source/WebKit/win/WebDownload.h 2013-07-22 16:35:15 UTC (rev 152976)
@@ -48,7 +48,7 @@
: public IWebDownload
, public IWebURLAuthenticationChallengeSender
#if USE(CURL)
-, public CurlDownloadListener
+, public WebCore::CurlDownloadListener
#endif
{
public:
@@ -141,7 +141,7 @@
#if USE(CFNETWORK)
RetainPtr<CFURLDownloadRef> m_download;
#elif USE(CURL)
- CurlDownload m_download;
+ WebCore::CurlDownload m_download;
#endif
COMPtr<IWebMutableURLRequest> m_request;
COMPtr<IWebDownloadDelegate> m_delegate;
Modified: trunk/Source/WebKit/win/WebDownloadCurl.cpp (152975 => 152976)
--- trunk/Source/WebKit/win/WebDownloadCurl.cpp 2013-07-22 16:26:43 UTC (rev 152975)
+++ trunk/Source/WebKit/win/WebDownloadCurl.cpp 2013-07-22 16:35:15 UTC (rev 152976)
@@ -67,10 +67,8 @@
m_delegate = delegate;
- KURL url(ParsedURLString, request.url());
+ m_download.init(this, handle, request, response);
- m_download.init(this, url);
-
start();
}