cert-find is a rather complex beast because it not only
looks for certificates in the optional CA but within the
IPA LDAP database as well. It has a process to deduplicate
the certificates since any PKI issued certificates will
also be associated with an IPA record.
In order to obtain the data to deduplicate the certificates
the cert from LDAP must be parser for issuer and serial number.
ipaldap has automation to determine the datatype of an
attribute and will use the ipalib.x509 IPACertificate class to
decode a certificate automatically if you access
entry['usercertificate'].
The downside is that this is comparatively slow. Here is the
parse time in microseconds:
cryptography 0.0081
OpenSSL.crypto 0.2271
ipalib.x509 2.6814
Since only issuer and subject are required there is no need to
make the expensive IPACertificate call.
The IPACertificate parsing time is fine if you're parsing one
certificate but if the LDAP search returns a lot of certificates,
say in the thousands, then those microseconds add up quickly.
In testing it took ~17 seconds to parse 5k certificates (excluding
transmission overhead, etc).
cert-find when there are a lot of certificates has been
historically slow. It isn't related to the CA which returns
large sets (well, 5k anyway) in a second or two. It was the
LDAP comparision adding tens of seconds to the runtime.
When searching with the default sizelimit of 100 the time is
~10s without this patch. With it the time is 1.5s.
CLI times from before and after searching for all certs:
original:
-------------------------------
Number of entries returned 5038
-------------------------------
real 0m15.507s
user 0m0.828s
sys 0m0.241s
using cryptography:
real 0m4.037s
user 0m0.816s
sys 0m0.193s
Fixes: https://pagure.io/freeipa/issue/9331
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
This reverts commit 1f30cc6527.
The problem isn't with python-cryptography, it is with the
IPACertificate class which does way more work on a certificate
than is necessary in cert-find.
Related: https://pagure.io/freeipa/issue/9331
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
This reverts commit 191880bc9f.
The problem isn't with python-cryptography, it is with the
IPACertificate class which does way more work on a certificate
than is necessary in cert-find.
Related: https://pagure.io/freeipa/issue/9331
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
When ipa cert-find --all is called, the function prints the
certificate public bytes. The code recently switched to OpenSSL.crypto
and the objects OpenSSL.crypto.X509 do not have the method
public_bytes(). Use to_cryptography() to transform into a
cryptography.x509.Certificate before calling public_bytes().
Related: https://pagure.io/freeipa/issue/9331
Signed-off-by: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
cert-find is a rather complex beast because it not only
looks for certificates in the optional CA but within the
IPA LDAP database as well. It has a process to deduplicate
the certificates since any PKI issued certificates will
also be associated with an IPA record.
In order to obtain the data to deduplicate the certificates
the cert from LDAP must be parser for issuer and serial number.
ipaldap has automation to determine the datatype of an
attribute and will use the python-cryptography engine to
decode a certificate automatically if you access
entry['usercertificate'].
The downside is that this is comparatively slow. Here is the
parse time in microseconds:
OpenSSL.crypto 175
pyasn1 1010
python-cryptography 3136
The python-cryptography time is fine if you're parsing one
certificate but if the LDAP search returns a lot of certificates,
say in the thousands, then those microseconds add up quickly.
In testing it took ~17 seconds to parse 5k certificates.
It's hard to overstate just how much better the cryptography
Python interface is. In the case of OpenSSL really the only
certificate fields easily available are serial number, subject
and issuer. And the subject/issuer are in the OpenSSL reverse
format which doesn't compare nicely to the cryptography format.
The DN module can correct this.
Fortunately for cert-find we only need serial number and issuer,
so the OpenSSL module fine. It takes ~2 seconds.
pyasn1 is also relatively faster but switch to it would require
subtantially more effort for less payback.
cert-find when there are a lot of certificates has been
historically slow. It isn't related to the CA which returns
large sets (well, 5k anyway) in a second or two. It was the
LDAP comparision adding tens of seconds to the runtime.
CLI times from before and after:
original:
-------------------------------
Number of entries returned 5011
-------------------------------
real 0m21.155s
user 0m0.835s
sys 0m0.159s
using OpenSSL:
real 0m5.747s
user 0m0.864s
sys 0m0.148s
OpenSSL is forcibly lazy-loaded so it doesn't conflict with
python-requests. See ipaserver/wsgi.py for the gory details.
Fixes: https://pagure.io/freeipa/issue/9331
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Antonio Torres <antorres@redhat.com>
The sizelimit option was not being passed into the dogtag
ra_find() command so it always returned all available certificates.
A value of 0 will retain old behavior and return all certificates.
The default value is the LDAP searchsizelimit.
Related: https://pagure.io/freeipa/issue/9331
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
Reviewed-By: Antonio Torres <antorres@redhat.com>
In IPA framework we don't properly convert to Python bool type and just
return a string (TRUE or FALSE). This can be seen with many boolean
attributes, like
Bool('idnsallowdynupdate?',
cli_name='dynamic_update',
label=_('Dynamic update'),
doc=_('Allow dynamic updates.'),
attribute=True,
default=False,
autofill=True
),
in 'ipa dnszone-show':
> > > api.Command.dnszone_show('ipa.test')['result']['idnsallowdynupdate']
['TRUE']
This is because we don't have the reverse (from LDAP to Python) mapping
for the LDAP boolean OID 1.3.6.1.4.1.1466.115.121.1.7.
When Web UI asks for the entry, it gets back JSON output that contains
this 'TRUE' value:
"idnsallowdynupdate": [
"TRUE"
],
Add proper mapping from LDAP to Python bool type. With this, a simple
'checkbox' type can be used in Web UI instead of a complex radio-box
setup.
Note that when IPA API is asked to return raw values, 'TRUE' and 'FALSE'
still returned. These are the actual LDAP boolean attribute values. Care
needs to be done in tests:
- if output is from a command with --raw option, 'TRUE' or 'FALSE'
should be expected
- if output if from a normal (non-raw) command, True or False would be
returned
Fixes: https://pagure.io/freeipa/issue/9171
Signed-off-by: Alexander Bokovoy <abokovoy@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Transmitting a big integer like a random serial number over
either xmlrpc or JSON is problematic because they only support
32-bit integers at best. A random serial number can be as big
as 128 bits (theoretically 160 but dogtag limits it).
Treat as a string instead. Internally the value can be treated
as an Integer to conversions to/from hex as needed but for
transmission purposes handle it as a string.
Fixes: https://pagure.io/freeipa/issue/2016
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Francisco Trivino <ftrivino@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
This reverts commit aa1350384a.
The search for certificates is a complex, three-step process,
which filters results in subsequent searches. This filters out
non-relevant certificates when deleting a host or service.
This patch breaks that so deleting one service of a host will
revoke *all* certificates for that host.
Another attempt will be made separately to implement this.
https://pagure.io/freeipa/issue/7835
Signed-off-by: Rob Crittenden rcritten@redhat.com
Reviewed-By: Florence Blanc-Renaud <frenaud@redhat.com>
This utilizes the new status option so that we only retrieve
VALID certificates when revoking certificates issued for a
specific host or service.
ae74d348c3 made a special case
in cert_find when searching for hosts and services so that if only
one host/service was searched on do a subject search. It only
works when there is exactly one option requested.
Since we want to restrict to only VALID certificates, pass
the hostname as subject directly when revoking certs when deleting
an entry to limit the number of certificates to consider.
https://pagure.io/freeipa/issue/7835
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
`dnspython` 2.0.0 has many changes and several deprecations like:
```
> dns.resolver.resolve() has been added, allowing control of whether
search lists are used. dns.resolver.query() is retained for backwards
compatibility, but deprecated. The default for search list behavior can
be set at in the resolver object with the use_search_by_default
parameter. The default is False.
> dns.resolver.resolve_address() has been added, allowing easy
address-to-name lookups.
```
The new class `DNSResolver`:
- provides the compatibility layer
- defaults the previous behavior (the search list configured in the
system's resolver configuration is used for relative names)
- defaults lifetime to 15sec (determines the number of seconds
to spend trying to get an answer to the question)
Fixes: https://pagure.io/freeipa/issue/8383
Signed-off-by: Stanislav Levin <slev@altlinux.org>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
The 'cert_request' command accumulates DNS names from the CSR,
before checking that all IP addresses in the CSR are reachable from
those DNS names. Before adding a DNS name to the set, we check that
that it corresponds to the FQDN of a known host/service principal
(including principal aliases). When a DNS name maps to a
"alternative" principal (i.e. not the one given via the 'principal'
argument), this check was not being performed correctly.
Specifically, we were looking for the 'krbprincipalname' field on
the RPC response object directly, instead of its 'result' field.
To resolve the issue, dereference the RPC response to its 'result'
field before invoking the '_dns_name_matches_principal' subroutine.
Fixes: https://pagure.io/freeipa/issue/8368
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Under some search conditions (in particular, when user is
specified), the CA sub-search of cert-find command throws an error
on CA-less deployments. Do not execute the CA sub-search on CA-less
deployments.
Fixes: https://pagure.io/freeipa/issue/8369
Reviewed-By: Christian Heimes <cheimes@redhat.com>
For detailed discussion on the purpose of this change and the design
decisions made, see `git log -1 $THIS_COMMIT~1`.
ACME support requires TLS and we want ACME clients to access the
service via the ipa-ca.$DOMAIN DNS name. So we need to add the
ipa-ca.$DOMAIN dNSName to IPA servers' HTTP certificates. To
facilitiate this, add a special case to the cert-request command
processing. The rule is:
- if the dnsName being validated is "ipa-ca.$DOMAIN"
- and the subject principal is an "HTTP/..." service
- and the subject principal's hostname is an IPA server
Then that name (i.e. "ipa-ca.$DOMAIN") is immediately allowed.
Otherwise continue with the usual dnsName validation.
Part of: https://pagure.io/freeipa/issue/8186
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Replace manual queries of container_masters with new APIs get_masters()
and is_service_enabled().
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
SAN IP address validation, while determining the zone for a DNS name
or IP address, does not handle missing zones. The resulting
dns.resolver.NoNameservers exception is not caught. As a result,
InternalError is returned to client.
Update cert-request IP address name validation to handle this case.
Part of: https://pagure.io/freeipa/issue/7451
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Update the IP address validation to raise different error messages
for:
- inability to reach IP address from a DNS name
- missing PTR records for IP address
- asymmetric PTR / forward records
If multiple scenarios apply, indicate the first error (from list
above).
The code should now be a bit easier to follow. We first build dicts
of forward and reverse DNS relationships, keyed by IP address. Then
we check that entries for each iPAddressName are present in both
dicts. Finally we check for PTR-A/AAAA symmetry.
Update the tests to check that raised ValidationErrors indicate the
expected error.
Part of: https://pagure.io/freeipa/issue/7451
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
During SAN validation, it is possible that more than one
iPAddressName does not match a known IP address for the DNS names in
the SAN. But only one unmatched IP address is reported. Update the
error message to mention all unmatched iPAddressName values.
Part of: https://pagure.io/freeipa/issue/7451
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Generalise _san_dnsname_ips to allow arbitrary cname depths. This
also clarifies the code and avoids boolean blindness. Update the
call site to maintain the existing behvaiour (one cname allowed).
Part of: https://pagure.io/freeipa/issue/7451
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Collect only qualified DNS names for IPAddress validation. This is
necessary because it is undecidable whether the name 'ninja' refers
to 'ninja.my.domain.' or 'ninja.' (assuming both exist). Remember
that even a TLD can have A records.
Now that we are only checking qualified names for the purpose of
IPAddressName validation, remove the name length hack from
_san_dnsname_ips().
Part of: https://pagure.io/freeipa/issue/7451
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
Allow issuing certificates with IP addresses in the subject
alternative name (SAN), if all of the following are true.
* One of the DNS names in the SAN resolves to the IP address
(possibly through a CNAME).
* All of the DNS entries in the resolution chain are managed by
this IPA instance.
* The IP address has a (correct) reverse DNS entry that is managed
by this IPA instance
https://pagure.io/freeipa/issue/7451
Reviewed-By: Florence Blanc-Renaud <flo@redhat.com>
host-del and service-del are slow because cert revokation is implemented
inefficiently. The internal cert_find() call retrieves all certificates
from Dogtag.
The workaround special cases service and host find without additional RA
search options. A search for service and host certs limits the scope to
certificate with matching subject common name.
See: https://pagure.io/freeipa/issue/7835
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Alexander Bokovoy <abokovoy@redhat.com>
Move LDAP service discovery and service definitions from
ipaserver.install to ipaserver. Simplify and unify different
implementations in favor of a single implementation.
Signed-off-by: Christian Heimes <cheimes@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
commit 7284097 (Delay enabling services until end of installer)
introduced a regression in replica installation.
When the replica requests a cert for PKINIT, a check is done
to ensure that the hostname corresponds to a machine with a
KDC service enabled (ipaconfigstring attribute of
cn=KDC,cn=<hostname>,cn=masters,cn=ipa,cn=etc,$BASEDN must contain
'enabledService').
With the commit mentioned above, the service is set to enabled only
at the end of the installation.
The fix makes a less strict check, ensuring that 'enabledService'
or 'configuredService' is in ipaconfigstring.
Fixes: https://pagure.io/freeipa/issue/7566
Reviewed-By: Christian Heimes <cheimes@redhat.com>
In the case that enabledService is not found ipaConfigString kdc entry, a
NotFound error was raised without setting the reason. This resulted in a
traceback.
Fixes: https://pagure.io/freeipa/issue/7652
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
In order to support pylint 2.0 the following violations must be fixed:
- `chained-comparison` (R1716):
Simplify chained comparison between the operands This message is
emitted when pylint encounters boolean operation like
"a < b and b < c", suggesting instead to refactor it to "a < b < c".
- `consider-using-in` (R1714):
Consider merging these comparisons with "in" to %r To check if a
variable is equal to one of many values,combine the values into a
tuple and check if the variable is contained "in" it instead of
checking for equality against each of the values.This is faster
and less verbose.
Issue: https://pagure.io/freeipa/issue/7614
Signed-off-by: Armando Neto <abiagion@redhat.com>
Reviewed-By: Christian Heimes <cheimes@redhat.com>
The command description is taken from python docstring. Thus
commands should have them and should include the callings of
gettext to be translated.
Reviewed-By: Christian Heimes <cheimes@redhat.com>
ipa cert_find command has an option called --subject.
The option is documented as --subject=STR Subject.
It is expected that a --subject option searches by X.509 subject field but it does not do so.
It searches for CN not cert subject. Hence changing content of --subject help option.
Resolves: https://pagure.io/freeipa/issue/7322
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
When executing cert-request, if Dogtag successfully issues a
certificate but python-cryptography cannot parse the certificate, an
unhandled exception occurs. Handle the exception by notifying about
the malformed certificate in the response messages.
Fixes: https://pagure.io/freeipa/issue/7390
Reviewed-By: Tibor Dudlak <tdudlak@redhat.com>
Change eb6d4c3037 deferred the
detailed lookup until all certs were collected but introduced
a bug where the ra backend was always retrieved. This generated a
backtrace in a CA-less install because there is no ra backend in
the CA-less case.
The deferral also removes the certificate value from the LDAP
search output resulting in only the serial number being displayed
unless --all is provided. Add a new class variable,
self.ca_enabled, to add an exception for the CA-less case.
Fixes https://pagure.io/freeipa/issue/7202
Signed-off-by: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Stanislav Laznicka <slaznick@redhat.com>
Previously, CSRs were handled as a Str parameter which brought
trouble to Python 3 because of its more strict type requirements.
We introduce a CertificateSigningRequest parameter which allows to
use python-cryptography x509.CertificateSigningRequest to represent
CSRs in the framework.
https://pagure.io/freeipa/issue/7131
Up until now, Bytes parameter was used for certificate parameters
throughout the framework. However, the Bytes parameter does nothing
special for certificates, like validation, so this had to be done
for each of the parameters which were supposed to represent a
certificate.
This commit introduces a special Certificate parameter which takes
care of certificate validation so this does not have to be done
separately. It also makes sure that the certificates represented by
this parameter are always converted to DER format so that we can work
with them in a unified manner throughout the framework.
This commit also makes it possible to pass bytes directly during
instantiation of the Certificate parameter and they are still
represented correctly after their conversion in the _convert_scalar()
method.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Splitting the load_certificate() function into two separate helps
us word the requirements for the input explicitly. It also makes
our backend similar to the one of python-cryptography so eventually
we can swap python-cryptography for IPA x509 module.
https://pagure.io/freeipa/issue/4985
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Reviewed-By: Rob Crittenden <rcritten@redhat.com>
Reviewed-By: Martin Basti <mbasti@redhat.com>
Remove all object-specific loggers, with the exception of `Plugin.log`,
which is now deprecated. Replace affected logger calls with module-level
logger calls.
Deprecate object-specific loggers in `ipa_log_manager.get_logger`.
Reviewed-By: Martin Basti <mbasti@redhat.com>
Currently the cert-request execution is complicated and cannot
handle aliases in the --principal argument.
Implement the following simplifications:
- Search all user/host/service accounts at once, by krbPrincipalName
(error if no account found). Use principal canonical name to
determine the type of the principal.
- Update subject principals userCertificate attribute uniformly,
instead of dispatching to user/host/service-mod based on type of
principal.
Fixes: https://fedorahosted.org/freeipa/ticket/6531
Reviewed-By: Felipe Volpone <felipevolpone@gmail.com>
In service.py the primary key is krbCanonicalName, which we
don't want to use to do searchs. Now, cert-find uses primary
key or a specified attribute to do searches in LDAP, instead
of using only a primary key.
https://pagure.io/freeipa/issue/6948
Reviewed-By: Martin Babinsky <mbabinsk@redhat.com>
Reviewed-By: Jan Cholasta <jcholast@redhat.com>
Reviewed-By: Fraser Tweedale <ftweedal@redhat.com>
Rob Crittenden
Florence Blanc-Renaud
Alexander Bokovoy
Stanislav Levin
Christian Heimes
Fraser Tweedale
Ian Pilcher
Thomas Woerner
Armando Neto
amitkuma
Stanislav Laznicka
Jan Cholasta
Felipe Volpone