Metadata-Version: 2.1
Name: pyasice
Version: 1.0
Summary: Manipulate ASiC-E containers and XAdES/eIDAS signatures for Estonian e-identity services
Home-page: https://github.com/thorgate/pyasice
Author: Thorgate
Author-email: yuriy@thorgate.eu
License: ISC
Description: # pyasice - ASiC-E (BDOC) and XAdES Manipulation Library
        
        The `pyasice` library is designed to:
        * create, read, and verify XAdES/XMLDsig/eIDAS electronic signatures,
        * validate signers' certificates with OCSP,
        * confirm these signatures with TimeStamping, 
        * create and manipulate [ASiC-E](https://en.wikipedia.org/wiki/Associated_Signature_Containers) or BDoc 2.1 containers, 
        which are based on the XAdES/eIDAS stack.  
        
        ## Contents
        
        * [Quickstart](#quickstart)
        * * [ASiC-E/BDOC Container File Manipulation](#asic-ebdoc-container-file-manipulation)
        * * [Signing Flow Utilities](#signing-flow-utilities)
        * [Normative References](#normative-references)
        * [Module Layout](#module-layout)
        * [Technology Stack](#technology-stack)
        * [Build the XAdES XML Signature meta-file](#build-the-xades-xml-signature-meta-file)
        * * [SignedInfo](#signedinfo)
        * * [SignatureValue](#signaturevalue)
        * * [KeyInfo](#keyinfo)
        * * [SignedProperties](#signedproperties)
        * [Secondary Services](#secondary-services)
        * * [OCSP](#ocsp)
        * * [Timestamping Service](#timestamping-service)
        
        ## Quickstart
        
        ### ASiC-E/BDOC Container File Manipulation
        
        Create a new container:
        ```python
        from pyasice import Container, XmlSignature
        
        xmlsig = XmlSignature.create().add_document('test.txt', b'Test data', 'application/pdf')
        # ... here goes the signing, confirming and timestamping part ... 
        
        container = Container()
        container\
            .add_file('test.txt', b'Test data', 'application/pdf')\
            .add_signature(xmlsig)\
            .save('test.asice')
        
        # container is a context manager:
        with Container() as container:
            container.add_file('a', b'b', 'c').save('path/to')
        
        # Open an existing container:
        container = Container.open('test.asice')
        
        # Verify container. Raises pyasice.SignatureVerificationError on failure
        container.verify_signatures()
        
        # Read files in the container
        with container.open_file('test.txt') as f:
            assert f.read() == b'Test data'
        
        # Iterate over signatures
        for xmlsig in container.iter_signatures():
            xmlsig.get_signing_time()
        ```
        
        ### Signing Flow Utilities
        
        ```python
        from pyasice import Container, finalize_signature
        
        # get this from an external service, ID card, or elsewhere
        user_certificate = b'user certificate in DER/PEM format'
        
        container = Container()
        container.add_file("test.txt", b'Test', "text/plain")
        
        xml_sig = container.prepare_signature(user_certificate)
        
        # Use an external service, or ID card, or a private key from elsewhere
        # to sign the XML signature structure
        signature_value = externally.sign(xml_sig.signed_data())
        xml_sig.set_signature_value(signature_value)
        
        # Get issuer certificate from the ID service provider, e.g. sk.ee. 
        # Here we use the user certificate's `issuer.common_name` field to identify the issuer cert,
        # and find the cert in the `esteid-certificates` PyPI package. 
        issuer_cert_name = xml_sig.get_certificate_issuer_common_name()
        import esteid_certificates
        issuer_certificate = esteid_certificates.get_certificate(issuer_cert_name)
        
        # Complete the XML signature with OCSP and optionally Timestamping
        finalize_signature(xml_sig, ocsp_url="https://ocsp.server.url", tsa_url="https://tsa.server.url")
        
        container.add_signature(xml_sig)
        
        container.save("path/to/file.asice")
        ```
        
        
        ## Normative References
        
        The main document this library is based on:
        the [BDOC 2.1.2 spec](https://www.id.ee/wp-content/uploads/2020/06/bdoc-spec212-eng.pdf).
        
        The specific standards outlined in that document:
        
        * [ETSI TS 101 903 v1.4.2](https://www.etsi.org/deliver/etsi_ts/101900_101999/101903/01.04.02_60/ts_101903v010402p.pdf) 
          – XML Advanced Electronic Signatures (XAdES) and its Baseline Profile ETSI TS 103 171;
        * ITU-T Recommendation X.509;
        * [RFC 3161](https://tools.ietf.org/html/rfc3161) – PKIX Time-Stamp protocol;
        * [RFC 6960](https://tools.ietf.org/html/rfc6960) – Online Certificate Status Protocol;
        * ETSI TS 102 918 v1.2.1 - Associated Signature Containers (ASiC) and its
          Baseline Profile ETSI TS 103 174.
        
        The difference between ASiC-E and BDOC is almost exclusively in terminology.
        
        The [BDOC 2.1.2 spec](https://www.id.ee/wp-content/uploads/2020/06/bdoc-spec212-eng.pdf) states:
        
        > The BDOC file format is based on ASiC standard which is in turn profiled by ASiC BP.
        > BDOC packaging is a ASiC-E XAdES type ZIP container ...
        
        So with a moderate risk of confusion, we can accept that ASiC-E and BDOC refer to the same thing.
        
        ## Module Layout
        
        * [container.py](container.py) -- the `Container` class, that deals with ASiC-E (BDOC v.2.1) container format 
        * [xmlsig.py](xmlsig.py) -- the `XmlSignature` class, that deals with XAdES/XMLDSig XML structures
        * [ocsp.py](ocsp.py) -- the `OCSP` class that deals with OCSP requests and responses
        * [tsa.py](tsa.py) -- the `TSA` class that deals with TimeStamping service requests and responses
        * [signature_verifier.py](signature_verifier.py) -- the `verify` function, to verify signatures against a certificate.
        
        ## Technology Stack
        
        Dealing with the subject involves, at least:
        * public key cryptography (RSA, ECDSA);
        * ASN.1 encoding;
        * XML processing;
        * Zip archives;
        * and also requests to various services (obtaining signer's certificate and the signature,
          validating the certificate through OCSP, time-stamping the signature).
        
        The [asn1crypto](https://github.com/wbond/asn1crypto) library and its higher-level complement 
        [oscrypto](https://github.com/wbond/oscrypto)
        allow handling certificates and ASN.1 structures quite easily.
        
        The [cryptography](https://cryptography.io/en/latest) library is by far the most powerful python library 
        for dealing with public key cryptography algorithms.
        
        
        ## Build the XAdES XML Signature meta-file
        
        The structure of the XAdES XML signature file looks like this:
        ```xml
        <asic:XAdESSignatures xmlns:asic="http://uri.etsi.org/02918/v1.2.1#" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
                              xmlns:xades="http://uri.etsi.org/01903/v1.3.2#">
            <ds:Signature Id="S0">
                <ds:SignedInfo Id="S0-SignedInfo">...</ds:SignedInfo>
                <ds:SignatureValue Id="S0-SIG">...</ds:SignatureValue>
                <ds:KeyInfo Id="S0-KeyInfo">...</ds:KeyInfo>
                <ds:Object Id="S0-object-xades">
                    <xades:QualifyingProperties Id="S0-QualifyingProperties" Target="#S0">
                        <xades:SignedProperties Id="S0-SignedProperties">
                            <xades:SignedSignatureProperties Id="S0-SignedSignatureProperties">
                                <xades:SigningTime>2019-06-07T14:03:50Z</xades:SigningTime>
                                <xades:SigningCertificate>...</xades:SigningCertificate>
                                <xades:SignaturePolicyIdentifer>...</xades:SignaturePolicyIdentifer>
                            </xades:SignedSignatureProperties>
                        </xades:SignedProperties>
                    </xades:QualifyingProperties>
                </ds:Object>
            </ds:Signature>
        </asic:XAdESSignatures>
        ```
        
        We'll go over each section below.
        
        * [SignedInfo](#signedinfo)
        * [SignatureValue](#signaturevalue)
        * [KeyInfo](#keyinfo)
        * [SignedProperties](#signedproperties)
        
        ### SignedInfo
        
        The `SignedInfo` node is the source of the data being signed. The XML content of the node, canonicalized
        using the `CanonicalizationMethod` as per the respective child node, is hashed using an algorithm defined in
        the `SignatureMethod` child node, and this hash is fed to a signing service (ID card, SmartID etc.) 
        
        ```xml
        <ds:SignedInfo Id="S0-SignedInfo">
            <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-c14n11"></ds:CanonicalizationMethod>
            <ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#ecdsa-sha256"></ds:SignatureMethod>
            <ds:Reference Id="S0-ref-0" URI="test.pdf">
                <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></ds:DigestMethod>
                <ds:DigestValue>...</ds:DigestValue>
            </ds:Reference>
            <ds:Reference Id="S0-ref-sp" Type="http://uri.etsi.org/01903#SignedProperties" URI="#S0-SignedProperties">
                <ds:Transforms>
                    <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/>
                </ds:Transforms>
                <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></ds:DigestMethod>
                <ds:DigestValue>...</ds:DigestValue>
            </ds:Reference>
        </ds:SignedInfo>
        ```
        
        The `Reference` fields are different in purpose and formation.
        
        The first `Reference` field is about the signed document and as such, has an `URI` attribute of the document's file name.
        Its child `DigestValue` element is the SHA256 hash of the document, it is, incidentally, the very hash that is sent to the SmartID API for signing.
        
        The second `Reference` is built on the basis of some fields defined later in the [SignedProperties](#SignedProperties) section.
        Its child `DigestValue` is calculated as a SHA256 hash of the canonicalized XML output of the  `SignedProperties` tag, after that one is formed:
        The `URI` attribute of this `Reference` tag is the `#`-prefixed `Id` attribute of the `SignedProperties` tag.
        
        ```python
        import base64
        import hashlib
        from lxml import etree
        
        buf = etree.tostring(el, method='c14n', exclusive=True or False)  # NOTE below
        digest_value = base64.b64encode(hashlib.sha256(buf).digest())
        ```
        (Assuming the `el` here to be the XML `<SignedProperties>` element)
        
        #### Canonicalization
        
        The `exclusive` kwarg controls whether the namespace declarations of ancestor tags should be included in the resulting canonical representation, or _excluded_.
        Whether to use `exclusive=True` depends on the canonicalization tag's `Algorithm` attribute: 
        * `http://www.w3.org/2001/10/xml-exc-c14n#`, uses `exclusive=True`,  
        * the two others, the required `http://www.w3.org/TR/2001/REC-xml-c14n-20010315`, or `http://www.w3.org/2006/12/xml-c14n11`, are not exclusive.
        
        The aforementioned `<ds:CanonicalizationMethod>` tag controls the c14n of the `SignedInfo` node before feeding its digest to the signature service. 
        The c14n of `SignedProperties` prior to getting its digest is determined by the `ds:Transform` tag within this `ds:Reference` node. 
        If it's not present, then the default, ie. not exclusive, c14n is used.  
        
        ### KeyInfo
        
        This section contains the base64-encoded user certificate value, e.g. the SmartID API response's `cert.value`,
        or the certificate obtained from an ID card:
        ```xml
        <ds:KeyInfo Id="S0-KeyInfo">
            <ds:X509Data>
                <ds:X509Certificate>MIIGJDCCBAygAwIBAgIQBNsLtTIpnmNbbE4+laSLaTANBgkqhkiG9w0BAQsFADBr...</ds:X509Certificate>
            </ds:X509Data>
        </ds:KeyInfo>
        ```
        More details about the certificate in the [SigningCertificate](#SigningCertificate) subsection.
         
        ### SignedProperties
        
        The XML section of `SignedProperties` consists of, [at least](https://www.w3.org/TR/XAdES/#IDAEAD1B), 
        the `SigningTime`, `SigningCertificate` and `SignaturePolicyIdentifer` elements.
        
        :question: The signatures returned by e.g. [Dokobit](https://dokobit.ee), 
        do not contain the `SignaturePolicyIdentifer` node.
        
        #### SigningTime
        
        A timestamp in ISO 8601 format.
        
        #### SignaturePolicyIdentifier 
        
        This appears to be a static^1 XML chunk referencing the BDOC 2.1 Specifications document:
        ```xml
        <xades:SignaturePolicyIdentifier>
            <xades:SignaturePolicyId>
                <xades:SigPolicyId>
                    <xades:Identifier Qualifier="OIDAsURN">urn:oid:1.3.6.1.4.1.10015.1000.3.2.1</xades:Identifier>
                </xades:SigPolicyId>
                <xades:SigPolicyHash>
                    <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256">
                    </ds:DigestMethod>
                    <ds:DigestValue>3Tl1oILSvOAWomdI9VeWV6IA/32eSXRUri9kPEz1IVs=</ds:DigestValue>
                </xades:SigPolicyHash>
                <xades:SigPolicyQualifiers>
                    <xades:SigPolicyQualifier>
                        <xades:SPURI>https://www.sk.ee/repository/bdoc-spec21.pdf</xades:SPURI>
                    </xades:SigPolicyQualifier>
                </xades:SigPolicyQualifiers>
            </xades:SignaturePolicyId>
        </xades:SignaturePolicyIdentifier>
        ```
        
        [1] The DigestValue is the hash value of the document referenced by `SPURI`, encoded in base64. 
        Refer to [BDOC 2.1:2014 Specification](https://www.id.ee/public/bdoc-spec212-eng.pdf) for more information.
        
        #### SigningCertificate
        
        The user certificate is a base64-encoded DER certificate which can be loaded as follows:
        ```python
        import base64
        from cryptography import x509
        from cryptography.hazmat.backends import default_backend
        cert_asn1 = base64.b64decode(cert_value)
        cert = x509.load_der_x509_certificate(base64.b64decode(cert_asn1), default_backend())
        ```
        or with `pyopenssl`:
        ```python
        import base64
        from OpenSSL.crypto import load_certificate, FILETYPE_ASN1
        cert_asn1 = base64.b64decode(cert_value)
        openssl_cert = load_certificate(FILETYPE_ASN1, base64.b64decode(cert_asn1))
        ```
        These objects expose a slightly different but similar API.
        
        What we need is the issuer name and certificate serial number:
        ```python
        assert openssl_cert.get_serial_number() == cert.sertial_number == '6454262457486410408874311107672836969'
        assert cert.issuer.rfc4514_string() == 'C=EE,O=AS Sertifitseerimiskeskus,2.5.4.97=NTREE-10747013,CN=TEST of ESTEID-SK 2015'
        assert openssl_cert.issuer.get_components() == [(b'C', b'EE'), (b'O', b'AS Sertifitseerimiskeskus'), (b'organizationIdentifier', b'NTREE-10747013'), (b'CN', b'ESTEID-SK 2015')]  
        ```
        
        Also we need a SHA256 digest value of the certificate:
        ```python
        cert_digest = base64.b64encode(hashlib.sha256(cert_asn1).digest())
        ```
        
        With these values we can build the certificate information entry of the SignedProperties:
        ```xml
        <xades:SigningCertificate>
            <xades:Cert>
                <xades:CertDigest>
                    <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></ds:DigestMethod>
                    <ds:DigestValue>hdsLTm4aaFKaGMwF6fvH5vWmiMBBnTCH3kba+TjY+pE=</ds:DigestValue>
                </xades:CertDigest>
                <xades:IssuerSerial>
                    <ds:X509IssuerName>C=EE,O=AS Sertifitseerimiskeskus,2.5.4.97=NTREE-10747013,CN=TEST of EID-SK 2016</ds:X509IssuerName>
                    <ds:X509SerialNumber>98652662091042892833248946646759285960</ds:X509SerialNumber>
                </xades:IssuerSerial>
            </xades:Cert>
        </xades:SigningCertificate>
        ```
        
        :question: Does `X509IssuerName` content need to be a `cert.issuer.rfc4514_string()` or can it be anything else?
        
        So, in the end, we get a `<xades:SignedProperties>` element which we then canonicalize and calculate a sha256 hash of this string,
        to place it in the appropriate `<ds:Reference>` element.
        
        ### SignatureValue
        
        ```xml
        <ds:SignatureValue Id="SIG-{SIGNATURE_ID}"><!-- Base64-encoded SIGNATURE_VALUE, gotten externally --></ds:SignatureValue>
        ```
        
        A base64-encoded value of the signature calculated over the signed data. 
        The signed data is the `ds:SignedInfo` section, as [described above](#signedinfo).
        
        When using SmartID/MobileID, this is taken from the `signature.value` field of the response.
        
        ### KeyInfo
        
        Contains the base64-encoded certificate, as gotten from the SmartID response.
        ```xml
        <ds:KeyInfo Id="S0-KeyInfo">
            <ds:X509Data>
                <ds:X509Certificate>...</ds:X509Certificate>
            </ds:X509Data>
        </ds:KeyInfo>
        ```
        
        ## Secondary Services
        
        ### OCSP
        
        OCSP ([Online Certificate Status Protocol](https://en.wikipedia.org/wiki/Online_Certificate_Status_Protocol)) 
        is designed to check that the signing certificate is valid at the point of signing. It is a binary protocol, and uses ASN.1 encoding in both request and response payload.
        To deal with it, we're using the `asn1crypto` library.
        
        The OCSP request should be made immediately after signing, and the base64-encoded response is embedded in the XAdES signature as a `xades:UnsignedSignatureProperties` descendant node,
        namely `xades:EncapsulatedOCSPValue`.
        
        #### Estonian eID
        
        URLs for OCSP services:
        * Demo: `http://demo.sk.ee/ocsp`
        * Production: `http://ocsp.sk.ee/`
        
        More detail on the [sk.ee OCSP page](https://www.sk.ee/en/services/validity-confirmation-services/technical-information/)
        
        
        ### Timestamping service
        
        The [TimeStamp protocol](https://en.wikipedia.org/wiki/Time_stamp_protocol) is also a binary protocol, for getting a Long-Term Validity Timestamp for a signature. 
        Also handled with the help of the `asn1crypto` library.
        
        The TSA request should be made immediately after OCSP validity confirmation, and the base64-encoded response is embedded in the XAdES signature as a `xades:UnsignedSignatureProperties` descendant node,
        namely `xades:EncapsulatedTimeStamp`.
        
        #### Estonian eID
        
        URLs for timestamping services:
        * Demo: `http://demo.sk.ee/tsa/`
        * Production: `http://tsa.sk.ee`
        
        More detail on the [sk.ee TSA page](https://www.sk.ee/en/services/time-stamping-service/)
        
Keywords: esteid asice xades smartid smart-id mobiilid mobile-id idcard
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: ISC License (ISCL)
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Description-Content-Type: text/markdown
