lidavidm commented on code in PR #177:
URL: https://github.com/apache/arrow-cookbook/pull/177#discussion_r847273922
##########
python/source/flight.rst:
##########
@@ -605,3 +605,138 @@ Or if we use the wrong credentials on login, we also get
an error:
server.shutdown()
.. _(HTTP) basic authentication:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme
+
+Authentication with certificates
+=================================
+
+Following on from the previous scenario where traffic to the server is managed
via a username and password,
+HTTPS (more specifically TLS) communication allows an additional layer of
security by encrypting messages
+between the client and server. This is achieved using certificates. During
development, the easiest
+approach is developing with self-signed certificates. At startup, the server
loads the public and private
+key and the client client authenticates itself to the server with the tls root
certificate.
+
+.. note:: In production environments it is recommended to make use of a
certificate signed by a certificate authority.
+
+**Step 1 - Generating the Self Signed Certificate**
+
+Generate a self-signed certificate by using dotnet on `Windows`_, or
`openssl`_ on Linux or MacOS.
+Alternatively, the self-signed certificate from the `Arrow testing data
repository`_ can be used.
+Depending on the file generated, you may need to convert it to a .crt and .key
file as required for the Arrow server.
+One method to achieve this is openssl, please visit this `IBM article`_ for
more info.
+
+
+**Step 2 - Running a server with TLS enabled**
+
+The code below is a minimal working example of an Arrow server used to receive
data with TLS. For a full server example, please visit the Arrow `GitHub
repo`_.
Review Comment:
We're in the middle of a bunch of examples, so I don't think we need to link
back to the GitHub repo here :slightly_smiling_face:
##########
python/source/flight.rst:
##########
@@ -605,3 +605,138 @@ Or if we use the wrong credentials on login, we also get
an error:
server.shutdown()
.. _(HTTP) basic authentication:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme
+
+Authentication with certificates
+=================================
+
+Following on from the previous scenario where traffic to the server is managed
via a username and password,
+HTTPS (more specifically TLS) communication allows an additional layer of
security by encrypting messages
+between the client and server. This is achieved using certificates. During
development, the easiest
+approach is developing with self-signed certificates. At startup, the server
loads the public and private
+key and the client client authenticates itself to the server with the tls root
certificate.
Review Comment:
This should be more like "...the client authenticates the server with the
TLS root certificate" right? In the usual flow the server does not authenticate
the client.
##########
python/source/flight.rst:
##########
@@ -605,3 +605,138 @@ Or if we use the wrong credentials on login, we also get
an error:
server.shutdown()
.. _(HTTP) basic authentication:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme
+
+Authentication with certificates
+=================================
Review Comment:
Since we're not talking about mTLS this should be more like "Securing
connections with TLS"
##########
python/source/flight.rst:
##########
@@ -605,3 +605,138 @@ Or if we use the wrong credentials on login, we also get
an error:
server.shutdown()
.. _(HTTP) basic authentication:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme
+
+Authentication with certificates
+=================================
+
+Following on from the previous scenario where traffic to the server is managed
via a username and password,
+HTTPS (more specifically TLS) communication allows an additional layer of
security by encrypting messages
+between the client and server. This is achieved using certificates. During
development, the easiest
+approach is developing with self-signed certificates. At startup, the server
loads the public and private
+key and the client client authenticates itself to the server with the tls root
certificate.
+
+.. note:: In production environments it is recommended to make use of a
certificate signed by a certificate authority.
+
+**Step 1 - Generating the Self Signed Certificate**
+
+Generate a self-signed certificate by using dotnet on `Windows`_, or
`openssl`_ on Linux or MacOS.
+Alternatively, the self-signed certificate from the `Arrow testing data
repository`_ can be used.
+Depending on the file generated, you may need to convert it to a .crt and .key
file as required for the Arrow server.
+One method to achieve this is openssl, please visit this `IBM article`_ for
more info.
+
+
+**Step 2 - Running a server with TLS enabled**
+
+The code below is a minimal working example of an Arrow server used to receive
data with TLS. For a full server example, please visit the Arrow `GitHub
repo`_.
+
+.. testcode::
+
+ import argparse
+ import pyarrow
+ import pyarrow.flight
+
+
+ class FlightServer(pyarrow.flight.FlightServerBase):
+ def __init__(self, host="localhost", location=None,
+ tls_certificates=None, verify_client=False,
+ root_certificates=None, auth_handler=None):
+ super(FlightServer, self).__init__(
+ location, auth_handler, tls_certificates, verify_client,
+ root_certificates)
+ self.flights = {}
+ self.host = host
+ self.tls_certificates = tls_certificates
Review Comment:
Are these ever used?
##########
python/source/flight.rst:
##########
@@ -605,3 +605,138 @@ Or if we use the wrong credentials on login, we also get
an error:
server.shutdown()
.. _(HTTP) basic authentication:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme
+
+Authentication with certificates
+=================================
+
+Following on from the previous scenario where traffic to the server is managed
via a username and password,
+HTTPS (more specifically TLS) communication allows an additional layer of
security by encrypting messages
+between the client and server. This is achieved using certificates. During
development, the easiest
+approach is developing with self-signed certificates. At startup, the server
loads the public and private
+key and the client client authenticates itself to the server with the tls root
certificate.
+
+.. note:: In production environments it is recommended to make use of a
certificate signed by a certificate authority.
+
+**Step 1 - Generating the Self Signed Certificate**
+
+Generate a self-signed certificate by using dotnet on `Windows`_, or
`openssl`_ on Linux or MacOS.
+Alternatively, the self-signed certificate from the `Arrow testing data
repository`_ can be used.
+Depending on the file generated, you may need to convert it to a .crt and .key
file as required for the Arrow server.
+One method to achieve this is openssl, please visit this `IBM article`_ for
more info.
+
+
+**Step 2 - Running a server with TLS enabled**
+
+The code below is a minimal working example of an Arrow server used to receive
data with TLS. For a full server example, please visit the Arrow `GitHub
repo`_.
+
+.. testcode::
+
+ import argparse
+ import pyarrow
+ import pyarrow.flight
+
+
+ class FlightServer(pyarrow.flight.FlightServerBase):
+ def __init__(self, host="localhost", location=None,
+ tls_certificates=None, verify_client=False,
+ root_certificates=None, auth_handler=None):
Review Comment:
Why not `**kwargs`?
##########
python/source/flight.rst:
##########
@@ -605,3 +605,138 @@ Or if we use the wrong credentials on login, we also get
an error:
server.shutdown()
.. _(HTTP) basic authentication:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme
+
+Authentication with certificates
+=================================
+
+Following on from the previous scenario where traffic to the server is managed
via a username and password,
+HTTPS (more specifically TLS) communication allows an additional layer of
security by encrypting messages
+between the client and server. This is achieved using certificates. During
development, the easiest
+approach is developing with self-signed certificates. At startup, the server
loads the public and private
+key and the client client authenticates itself to the server with the tls root
certificate.
+
+.. note:: In production environments it is recommended to make use of a
certificate signed by a certificate authority.
+
+**Step 1 - Generating the Self Signed Certificate**
+
+Generate a self-signed certificate by using dotnet on `Windows`_, or
`openssl`_ on Linux or MacOS.
+Alternatively, the self-signed certificate from the `Arrow testing data
repository`_ can be used.
+Depending on the file generated, you may need to convert it to a .crt and .key
file as required for the Arrow server.
+One method to achieve this is openssl, please visit this `IBM article`_ for
more info.
+
+
+**Step 2 - Running a server with TLS enabled**
+
+The code below is a minimal working example of an Arrow server used to receive
data with TLS. For a full server example, please visit the Arrow `GitHub
repo`_.
+
+.. testcode::
+
+ import argparse
+ import pyarrow
+ import pyarrow.flight
+
+
+ class FlightServer(pyarrow.flight.FlightServerBase):
+ def __init__(self, host="localhost", location=None,
+ tls_certificates=None, verify_client=False,
+ root_certificates=None, auth_handler=None):
+ super(FlightServer, self).__init__(
+ location, auth_handler, tls_certificates, verify_client,
+ root_certificates)
+ self.flights = {}
+ self.host = host
+ self.tls_certificates = tls_certificates
+
+ @classmethod
+ def descriptor_to_key(self, descriptor):
+ return (descriptor.descriptor_type.value, descriptor.command,
+ tuple(descriptor.path or tuple()))
+
+ def do_put(self, context, descriptor, reader, writer):
+ key = FlightServer.descriptor_to_key(descriptor)
+ print(key)
+ self.flights[key] = reader.read_all()
+ print(self.flights[key])
+
+
+ def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--tls", nargs=2, default=None,
metavar=('CERTFILE', 'KEYFILE'))
+ args = parser.parse_args()
+ tls_certificates = []
+
+ scheme = "grpc+tls"
+ host = "localhost"
+ port = "5005"
+
+ with open(args.tls[0], "rb") as cert_file:
+ tls_cert_chain = cert_file.read()
+ with open(args.tls[1], "rb") as key_file:
+ tls_private_key = key_file.read()
+
+ tls_certificates.append((tls_cert_chain, tls_private_key))
+
+ location = "{}://{}:{}".format(scheme, host, port)
+
+ server = FlightServer(host, location,
+ tls_certificates=tls_certificates)
+ print("Serving on", location)
+ server.serve()
+
+
+ if __name__ == '__main__':
+ main()
+
+Running the server, you should see ``Serving on grpc+tls://localhost:5005``.
+
+**Step 3 - Securely Connecting to the Server**
+Suppose we want to connect to the client and push some data to it. The
following code securely sends information to the server using TLS encryption.
+The example below shows how one could
Review Comment:
this looks to have been cut off?
##########
python/source/flight.rst:
##########
@@ -605,3 +605,138 @@ Or if we use the wrong credentials on login, we also get
an error:
server.shutdown()
.. _(HTTP) basic authentication:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme
+
+Authentication with certificates
+=================================
+
+Following on from the previous scenario where traffic to the server is managed
via a username and password,
+HTTPS (more specifically TLS) communication allows an additional layer of
security by encrypting messages
+between the client and server. This is achieved using certificates. During
development, the easiest
+approach is developing with self-signed certificates. At startup, the server
loads the public and private
+key and the client client authenticates itself to the server with the tls root
certificate.
+
+.. note:: In production environments it is recommended to make use of a
certificate signed by a certificate authority.
+
+**Step 1 - Generating the Self Signed Certificate**
+
+Generate a self-signed certificate by using dotnet on `Windows`_, or
`openssl`_ on Linux or MacOS.
+Alternatively, the self-signed certificate from the `Arrow testing data
repository`_ can be used.
+Depending on the file generated, you may need to convert it to a .crt and .key
file as required for the Arrow server.
+One method to achieve this is openssl, please visit this `IBM article`_ for
more info.
+
+
+**Step 2 - Running a server with TLS enabled**
+
+The code below is a minimal working example of an Arrow server used to receive
data with TLS. For a full server example, please visit the Arrow `GitHub
repo`_.
+
+.. testcode::
+
+ import argparse
+ import pyarrow
+ import pyarrow.flight
+
+
+ class FlightServer(pyarrow.flight.FlightServerBase):
+ def __init__(self, host="localhost", location=None,
+ tls_certificates=None, verify_client=False,
+ root_certificates=None, auth_handler=None):
+ super(FlightServer, self).__init__(
+ location, auth_handler, tls_certificates, verify_client,
+ root_certificates)
+ self.flights = {}
+ self.host = host
+ self.tls_certificates = tls_certificates
+
+ @classmethod
+ def descriptor_to_key(self, descriptor):
+ return (descriptor.descriptor_type.value, descriptor.command,
+ tuple(descriptor.path or tuple()))
+
+ def do_put(self, context, descriptor, reader, writer):
+ key = FlightServer.descriptor_to_key(descriptor)
+ print(key)
+ self.flights[key] = reader.read_all()
+ print(self.flights[key])
+
+
+ def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--tls", nargs=2, default=None,
metavar=('CERTFILE', 'KEYFILE'))
+ args = parser.parse_args()
+ tls_certificates = []
+
+ scheme = "grpc+tls"
+ host = "localhost"
+ port = "5005"
+
+ with open(args.tls[0], "rb") as cert_file:
+ tls_cert_chain = cert_file.read()
+ with open(args.tls[1], "rb") as key_file:
+ tls_private_key = key_file.read()
+
+ tls_certificates.append((tls_cert_chain, tls_private_key))
+
+ location = "{}://{}:{}".format(scheme, host, port)
+
+ server = FlightServer(host, location,
+ tls_certificates=tls_certificates)
+ print("Serving on", location)
+ server.serve()
+
+
+ if __name__ == '__main__':
+ main()
+
+Running the server, you should see ``Serving on grpc+tls://localhost:5005``.
+
+**Step 3 - Securely Connecting to the Server**
+Suppose we want to connect to the client and push some data to it. The
following code securely sends information to the server using TLS encryption.
+The example below shows how one could
+
+.. testcode::
+
+ import argparse
+ import pyarrow
+ import pyarrow.flight
+ import pandas as pd
+
+ # Assumes incoming data object is a Dataframe
Review Comment:
```suggestion
# Assumes incoming data object is a Pandas DataFrame
```
##########
python/source/flight.rst:
##########
@@ -605,3 +605,138 @@ Or if we use the wrong credentials on login, we also get
an error:
server.shutdown()
.. _(HTTP) basic authentication:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication#basic_authentication_scheme
+
+Authentication with certificates
+=================================
+
+Following on from the previous scenario where traffic to the server is managed
via a username and password,
+HTTPS (more specifically TLS) communication allows an additional layer of
security by encrypting messages
+between the client and server. This is achieved using certificates. During
development, the easiest
+approach is developing with self-signed certificates. At startup, the server
loads the public and private
+key and the client client authenticates itself to the server with the tls root
certificate.
+
+.. note:: In production environments it is recommended to make use of a
certificate signed by a certificate authority.
+
+**Step 1 - Generating the Self Signed Certificate**
+
+Generate a self-signed certificate by using dotnet on `Windows`_, or
`openssl`_ on Linux or MacOS.
+Alternatively, the self-signed certificate from the `Arrow testing data
repository`_ can be used.
+Depending on the file generated, you may need to convert it to a .crt and .key
file as required for the Arrow server.
+One method to achieve this is openssl, please visit this `IBM article`_ for
more info.
+
+
+**Step 2 - Running a server with TLS enabled**
+
+The code below is a minimal working example of an Arrow server used to receive
data with TLS. For a full server example, please visit the Arrow `GitHub
repo`_.
+
+.. testcode::
+
+ import argparse
+ import pyarrow
+ import pyarrow.flight
+
+
+ class FlightServer(pyarrow.flight.FlightServerBase):
+ def __init__(self, host="localhost", location=None,
+ tls_certificates=None, verify_client=False,
+ root_certificates=None, auth_handler=None):
+ super(FlightServer, self).__init__(
+ location, auth_handler, tls_certificates, verify_client,
+ root_certificates)
+ self.flights = {}
+ self.host = host
+ self.tls_certificates = tls_certificates
+
+ @classmethod
+ def descriptor_to_key(self, descriptor):
+ return (descriptor.descriptor_type.value, descriptor.command,
+ tuple(descriptor.path or tuple()))
+
+ def do_put(self, context, descriptor, reader, writer):
+ key = FlightServer.descriptor_to_key(descriptor)
+ print(key)
+ self.flights[key] = reader.read_all()
+ print(self.flights[key])
+
+
+ def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument("--tls", nargs=2, default=None,
metavar=('CERTFILE', 'KEYFILE'))
+ args = parser.parse_args()
+ tls_certificates = []
+
+ scheme = "grpc+tls"
+ host = "localhost"
+ port = "5005"
+
+ with open(args.tls[0], "rb") as cert_file:
+ tls_cert_chain = cert_file.read()
+ with open(args.tls[1], "rb") as key_file:
+ tls_private_key = key_file.read()
+
+ tls_certificates.append((tls_cert_chain, tls_private_key))
+
+ location = "{}://{}:{}".format(scheme, host, port)
+
+ server = FlightServer(host, location,
+ tls_certificates=tls_certificates)
+ print("Serving on", location)
+ server.serve()
+
+
+ if __name__ == '__main__':
+ main()
+
+Running the server, you should see ``Serving on grpc+tls://localhost:5005``.
+
+**Step 3 - Securely Connecting to the Server**
+Suppose we want to connect to the client and push some data to it. The
following code securely sends information to the server using TLS encryption.
+The example below shows how one could
+
+.. testcode::
+
+ import argparse
+ import pyarrow
+ import pyarrow.flight
+ import pandas as pd
+
+ # Assumes incoming data object is a Dataframe
+ def push_to_server(name, data, client):
+ objectToSend = pyarrow.Table.from_pandas(data)
Review Comment:
nit: stick to `snake_case` in Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
