deft/notes/cisco_ft_securex_registration.org

:PROPERTIES:
:ID:       1208f09c-d37d-4e6b-9110-151f3c6b7d34
:END:
#+TITLE: Cisco FT SecureX Simplified Registration
#+Author: Yann Esposito
#+Date: [2021-12-07]
#+OPTIONS: prop:t

- tags :: [[id:299643a7-00e5-47fb-a987-3b9278e89da3][Auth]]
- ref :: https://github.com/advthreat/response/issues/821
- source :: https://github.com/advthreat/iroh/issues/6076
- dashboard :: https://github.com/advthreat/iroh/projects/32
- design ::  https://www.figma.com/file/Bz3m25kpWXpdct7AnhmNsW/SXSO-Registeration?node-id=757%3A4369

* Table of Content :TOC_4:QUOTE:
#+BEGIN_QUOTE
- [[#functional-spec][Functional Spec]]
- [[#17-technical-plan][=17= Technical Plan]]
  - [[#0-support-private-email-vs-public-emails][=0= Support private email vs public emails]]
    - [[#canceled-support-allow-list-exceptions-for-some-cisco-user][CANCELED Support allow-list exceptions for some Cisco user.]]
  - [[#1-support-search-admin-with-same-email-domain][=1= Support, search admin with same email domain]]
  - [[#12-new-ui-sync][=12= New UI Sync]]
    - [[#1-new-auth-apis][=1= New Auth APIs]]
      - [[#tasks][Tasks]]
    - [[#1-new-api-jwt-middleware][=1= New API JWT middleware]]
    - [[#5-new-iroh-auth-api-endpoints][=5= New IROH-Auth API endpoints]]
      - [[#4-orgaccessrequestservice][=4= ~OrgAccessRequestService~]]
      - [[#1-optional-useridentityservice][=1= /OPTIONAL/ ~UserIdentityService~]]
    - [[#4-new-iroh-auth-api][=4= New IROH-Auth API]]
      - [[#whoami-endpoint][=whoami= Endpoint]]
      - [[#create-new-account][Create New Account]]
      - [[#canceled-list-matching-accounts][CANCELED List Matching Accounts]]
      - [[#list-pending-invites][List Pending Invites]]
      - [[#request-org-access][Request Org Access]]
      - [[#renew-org-access-request][Renew Org Access Request]]
      - [[#list-matching-orgs][List Matching Orgs]]
      - [[#registration-view][Registration View]]
      - [[#canceled-optional-hide-matching-org][CANCELED /OPTIONAL/ Hide Matching Org]]
      - [[#redirects-user][Redirects User]]
    - [[#1-optional-sync-useridentity-on-all-successful-securex-login][=1= /OPTIONAL/ Sync UserIdentity on all successful SecureX login]]
  - [[#3-email-notification-of-org-request-accesses][=3= Email Notification of Org Request Accesses]]
    - [[#notes][Notes]]
  - [[#2-org-requests-crud-api-for-admins-of-the-orgs][=2= Org Requests CRUD API for Admins of the Orgs]]
    - [[#list][List]]
    - [[#read][Read]]
    - [[#patch-the-org-access][Patch the Org Access]]
  - [[#3-optional-email-templating-service--interface][=3= /OPTIONAL/ Email Templating Service & Interface]]
    - [[#dev-details][Dev details]]
- [[#notes-1][Notes]]
  - [[#number-of-user-with-a-matching-domain-email-by-domain][Number of user with a matching domain email by domain]]
  - [[#number-of-orgs-with-more-than-100-users][Number of orgs with more than 100 users]]
  - [[#number-of-orgs-by-email-domain-name][Number of orgs by email domain name]]
  - [[#number-of-matching-orgs-per-email-domain-name][Number of matching orgs per email domain name]]
  - [[#number-of-admins-per-org][Number of admins per org]]
#+END_QUOTE

* Functional Spec

Response issues:
- https://github.com/advthreat/response/issues/821

Figma: https://www.figma.com/file/Bz3m25kpWXpdct7AnhmNsW/SXSO-Registeration?node-id=759%3A5926

* =17= Technical Plan
/Estimate: 17 rcd/

- rcd :: rcd stands for release cycle for 1 dev

** DONE =0= Support private email vs public emails

*DONE*
+/Estimate: 1 rcd after the list is provided./+

The solution is to use a blacklist of domains where any user could create
multiple email accounts pseudo-anonymously.

Details: https://github.com/advthreat/response/issues/979

*** CANCELED Support allow-list exceptions for some Cisco user.
:LOGBOOK:
- State "CANCELED"   from "HOLD"       [2022-01-17 Mon 10:57] \\
  This only concern new account creation. User with gmail account could still login.
:END:

/Estimate: 1 rcd after the list is provided./

Typically we should allow some users with an email like =some-user+XXX@gmail.com=

** =1= Support, search admin with same email domain

/Estimate: 1 rcd./
Note: *Work in progress*

We should be able given an email from a user, to find all the orgs for
which at least one of its admin has a matching domain name.

1. Most efficient: add an invisible field =email-domain= to all users. This
   should be lower-case, and we will need a migration.
   Doing this we could have a faster match than using string related queries.

Problems, users can login in the same user, with the same public email with
different emails.
This should be rare.

2. Search via text match.


The algorithm should look a bit like:

#+begin_src clojure

;; only when this is an unknown user
;; so a single approval will prevent the user to see this page.
(let [user-email ,,,
      domain (string/replace user-email #".*@" "")
      users (matching-admins domain) ;; returns a potentially big list of admin users
      indexed-orgs (group-by :org-id users)]
  (vals indexed-orgs))
#+end_src


Once this list of orgs is found.
We should also check the list of pending or rejected OrgAccessRequest for this user in
order to prevent the user to request access multiple time.

** =12= New UI Sync

/Estimate: 12 rcd/

We should find a way to hand the UI work to the UI team.
Right now, the page are all generated in IROH.

To reach that ideally we should sync the source code as a jar in IROH.

In order to give the UI the ability to make a front-end application, we
should create news APIS that support a new UserIdentity-level JWT

*** =1= New Auth APIs
/Estimate: 1 rcd/

ref :: https://github.com/advthreat/iroh/issues/6242

Continue to use the =/code= route of iroh-auth.
But instead of returning a Session Token, it will return a UserIdentity Token.
This is a JWT with the important data from the IdP:

- idp-id
- user-identity-id
- user-email
- etc…

When the user is redirected to an HTML generated page, we should add the
=code= in the anchor parameter of the route so the UI will be able to use
that code to retrieve a UserIdentity Token.
We should derive the current mechanism with code, but use the login code store.

**** Tasks

- Create a new function to generate these new JWT using the "token-infos"

*** =1= New API JWT middleware
/Estimate: 1 rcd/
- ref :: https://github.com/advthreat/iroh/issues/6266

We need to change the configuration of the check JWT middleware to support
UserIdentity Token instead. And use this configuration for this new API.

The =user-identity-jwt= should contain enough data to retrieve:
- =idp-mapping=
- =user-email=
- all other metas, as user-name, user-nick, etc…

*** =5= New IROH-Auth API endpoints
/Estimate: 5 rcd/
- ref :: https://github.com/advthreat/iroh/issues/6268

**** =4= ~OrgAccessRequestService~

/Estimate: 4 rcd/

- ref :: https://github.com/advthreat/iroh/issues/6272

This new service should be mostly a CRUD service on top of =TKStore= with the
following schema:

#+begin_src clojure
(s/defschema OrgAccessRequestStatus
  (s/enum :pending :accepted :rejected))

(s/defschema OrgAccessRequest
  (st/merge
   {:id UUID
    :secret s/Str ;; should be generated via `iroh-auth.lib.security/password`
    :idp-mapping IdPMapping
    :user-email s/Str
    :org-id s/Str
    :status OrgAccessRequestStatus
    :created-at DateTime}
   (st/optional-keys
    {:user-name s/Str
     :user-nick s/Str
     :granted-role Role ;; the role granted if the request is accepted
     :approver-id UserId
     :approver-email UserEmail ;; email of the approver
     :updated-at DateTime
     })))
#+end_src

#+begin_src clojure
(defprotocol OrgAccessRequestService
  "See iroh-auth.registration.org-access-request.schemas/ServiceFns for schemas."
  ;; Internal CRUD+Search almost only use iroh-crud
  (raw-search-org-access-requests
    [this filter-map pagination-params]
    "Search all OrgAccessRequest grants")

  (raw-get-org-access-request
    [this org-access-request-id]
    "Return the OrgAccessRequest grant")

  (raw-patch-org-access-request
    [this org-access-request-id org-access-request-patch]
    "Update the status of an OrgAccessRequest.")

  ;; service function for the Admins logged in SecureX
  ;; User filtered CRUD+Search for REST API related methods
  ;; The first argument is a RequestIdentity (:identity request) generated from a normal JWT
  ;; via the ring-jwt-middleware.
  (search-org-access-requests
    [this request-identity filter-map pagination-params]
    "Search all OrgAccessRequest of the org of the user of the request-identity")

  (get-org-access-request
    [this request-identity org-access-request-id]
    "Return the OrgAccessRequest for a user using the org-access-request-id")

  (patch-org-access-request
    [this request-identity org-access-request-id org-access-request-patch]
    "Change the status of an OrgAccessRequest to grant/reject it.
     Note user creation could be a side effect.")

  ;; For the New Registration Page (the user logged in via the IdP successfully)
  ;; The first argument is a UserIdentity (:identity request) it will not contain
  ;; any data related to any org
  (search-org-access-requests-for-user-identity
    [this user-identity filter-map pagination-params]
    "Search all OrgAccessRequest made by this user identity accross all orgs.
     This should only be visible via the registration page on the new API accepting
     user-identity JWT")

  (create-org-access-request
    [this user-identity org-id]
    "Create a new OrgAccessRequest. Should potentially send emails along the way")

  (renew-org-access-request
    [this user-identity org-id]
    "Renew an OrgAccessRequest. Mainly should send the email again and change the updated-at date."))
#+end_src
***** =1= search/get/patch
/Estimate: 1 rcd/
***** =1= Create

For the =create-org-access-request= an email should be send to the oldest
active admin of the matching org.

***** =3= PATCH
/Estimate: 3 rcd/

The org-access-request-patch should have the following schema:

#+begin_src clojure
{(s/optional-key :role) Role
 :status (s/enum :accepted :rejected)}
#+end_src

If the role is not provided, the default value should be =user=.
If the old ~OrgAccessRequest~ is not ~:pending~ this should throw a 400.
If the status is set to =:accepted=:

1. create a new user for the org of the ~OrgAccessRequest~.
2. send an email to the user that requested the access

If the status is set to =:rejected=, then send an email to the user that
requested the access.

Note we should test that the user created that way could login correctly.
***** =1= Unauthenticated PATCH
/Estimate: 1 rcd/

If possible (if we do not have any dependency cycle issue), it would be
nice to put this new route under ~/iroh/iroh-auth~ (so ~IROHAuthWebService~)
because this web service already supports access to non-logged-in users.

So have a new ~GET~ endpoint; ideally:

#+begin_src http
POST /iroh/iroh-auth/org-access-request-approval?code=ENCRYPTED_DATA&role=ROLE
#+end_src

Where ~ENCRYPTED_DATA~ should contain the following:

- ~org-access-request-id~
- ~org-access-request-secret~
- ~action~ ; either =accepted= or =rejected=
- ~approver-id~ ; user-id of the approver

and

- ~role~; optional and only if accepted, if none provided, default to ~user~

The separation of the ~role~ query parameter is important in order to give a
chance for the UI to change the associated role before user creation.
But once the user is created with a specified ~role~ it should be impossible
to change its role from an unauthenticated route.

If someone hit this endpoint, then you should:

1. Decrypt the ~code~ and retrieve the data.
2. Check the ~org-access-request~ id and secret matches a pending org access request.
   If not, return the corresponding error message.
3. check that ~approver-id~ is a ~user-id~ of an admin of the ~org~ of the ~org-access-request~.
4. If everything is fine, perform a patch that could potentially create a new user.

In order to make this work, you should probably have some work to do in the
email creation.
Instead of letting the URL be built inside the template, you should instead
provide a data structure with 3 links:

- ~accept-as-user-url~
- ~accept-as-admin-url~
- ~reject-url~

And pass that to the template that will be able to be used with:

#+begin_src html
<li><a href="{{accept-as-user-url}}">Grant access as User</a></li>
<li><a href="{{accept-as-admin-url}}">Grant access as Admin</a></li>
<li><a href="{{reject-url}}">Reject</a></li>
#+end_src

#+begin_src plantuml :file unauth-account.png
title Unauthenticated Account creation

User -> SXSO: login
SXSO -> Registration UI:
User -> Registration UI: create OrgRequestAccess (admin receive email)
Admin -> Mail: click on accept link
User -> Mail: receive

#+end_src


****** UI

From the backend, we need a path where a page will be displayed.
We should create links in the email that will look like this:

#+begin_src
<IROH_URL>/<path-to-approval-page>?code=ENCRYPTED_DATA&role=ROLE
#+end_src

That UI page should parse the URL and get both the ~code~ and the ~role~.
The ~role~ query parameter should be optional. By default, it should be ~user~.

If anyone ends up on this page, there should be a POST to the
~org-access-request-approval~ endpoint mentioned.

Once the call is made, there is no mechanism to change the attributed role.

**** =1= /OPTIONAL/ ~UserIdentityService~

/Estimate: 1 rcd/

- ref :: https://github.com/advthreat/iroh/issues/6274

Create a new service dedicated to ~UserIdentities~.

#+begin_src clojure
(defprotocol UserIdentityService
  "Service which handle UserIdentities (mostly CRUD operations). "
  (create-user-identity [this user-params])
  (get-user-identity [this user-id])
  (update-user-identity [this user-params])
  (update-login-date [this user-id])
  (search-user-identities
    [this filter-map pagination-params]
    [this filter-map pagination-params stringmatch-query-params])
  (delete-user-identity [this user-id]))
#+end_src

It should also contain a field containing a set of hidden org-ids that will
be the orgs not to display on the registration webpages.

#+begin_src clojure
(s/defschema UserIdentity
  (ist/open-schema-any-keys
   {:id s/Str
    :email
    :name
    :nickname
    :last-logged-in [,,,]
    (s/optional-key :hidden-orgs) #{,,,}}))
#+end_src

*** =4= New IROH-Auth API
/Estimate: 4 rcd/
- ref :: https://github.com/advthreat/iroh/issues/6268

/Depends: IROH-Auth CRUD Service/

This new API should only work with UserIdentity JWT.

#+begin_src
GET  /iroh/iroh-auth-ui/whoami ;; whoami at the User Identity level
POST /iroh/iroh-auth-ui/org-access/:org-id ;; request access to a matching org
POST /iroh/iroh-auth-ui/hide-org/:org-id ;; ability to hide an org-access
GET  /iroh/iroh-auth-ui/matching-accounts ;; list all the matching accounts
GET  /iroh/iroh-auth-ui/matching-orgs ;; list all the matching orgs
GET  /iroh/iroh-auth-ui/pending-invites ;; list all the pending invites
GET  /iroh/iroh-auth-ui/registration-view ;; helper to make a single http call
#+end_src

**** =whoami= Endpoint

- ref :: https://github.com/advthreat/iroh/issues/6266

Write an endpoint that would use the ~UserIdentity~ JWT and decode it.
Ideally should instead read the value in DB via the ~UserIdentityService~.
**** Create New Account
/Estimate: 2 rcd/
- ref :: https://github.com/advthreat/iroh/issues/6280

Create a new endpoint to create a new account (user + org):

#+begin_src http
GET  /iroh/iroh-auth-ui/create-new-account

Accept: application/json
Content-Type: application/json
User-Agent: ob-http
Authorization: Bearer ${user-identity-jwt}

{"org-name":"Cisco",
 "country":"US"}
#+end_src

The clojure code for the route should roughly look like:

#+begin_src clojure
(s/defschema NewAccount
  (st/merge
   {:org-name :- s/Str
    :country :- (apply s/enum country-iso-codes)
    (st/optional-keys
     {:department :- s/Str
      :street1 :- s/Str
      :street2 :- s/Str
      :postal-code :- s/Str
      :city :- s/Str})}))

(POST "/create-new-account" []
  :summary "Given a code and some org-settings create a new account (new org and new user)"
  :description "This is an internal temporary route needed to select the user/org."
  :body  [{:keys [country
                  org-name
                  department
                  street1
                  street2
                  postal-code
                  city] :as  new-account} NewAccount]
  (let [address (iroh-core/assoc-some?
                 {:country-iso-code country}
                 :department department
                 :street1 street1
                 :street2 street2
                 :postal-code postal-code
                 :city city)
        org-settings {:name org-name
                      :address address}]
    (resp/found (create-new-account org-settings))))
#+end_src


As we now have a session, we should take care about a few details:

***** Important Security Remarks

****** Should we keep track of the =origin=?

When a user login there is always an =origin= parameter (by default it will
point to SecureX).
But a user can login and create an account and should ultimately be
redirected to the correct URL (product).
Currently, the user can be redirected to:

- SecureX
- CTR
- Orbital

We should put the =origin= in the JWT and we should take care that the origin
has been checked via the =allowed-login-origins=.

The question about should we return a 301 to the =create-new-account= is open.
But this is probably a good idea to let only the backend take care of the
security of the redirection.
Note this is a very important security concern otherwise an attack in the
redirection code might send the credentials to an attacker quite easily.
****** Should we prevent a user identity to create multiple accounts?

I don't think so.
Not in the first round at least.
It will probably be easy to add a =created-by= metas in the org, and prevent
duplicates (or put a maximal number of authorized enabled orgs)
**** CANCELED List Matching Accounts
- ref :: https://github.com/advthreat/iroh/issues/6270

#+begin_src http GET
/iroh/iroh-auth-ui/matching-accounts

Accept: application/json
Content-Type: application/json
User-Agent: ob-http
Authorization: Bearer ${user-identity-jwt}
#+end_src

It should return a list of object with the following schema sorted by last
logged in time by the user:

#+begin_src clojure
{:user User
 :org Org
 :last-login-date DateTime
 :relative-last-login s/Str
 :org-created-at DateTime
 :relative-org-created-at s/Str
 :activated? s/Bool
 :org-nb-users s/Int}
#+end_src

Mainly should return the data structure used in the current selection
account page using similar order and functionalities.
**** DONE List Pending Invites

- ref :: https://github.com/advthreat/iroh/issues/6269

#+begin_src http
GET  /iroh/iroh-auth-ui/pending-invites
#+end_src

Here is an example value:

#+begin_src clojure
[{:role "admin",
  :org-id "org-1",
  :expires-in-nb-days 7,
  :status "pending",
  :invitee-email "chuck@example.org",
  :inviter-user-id "org-1-admin-1"}]
#+end_src

We already retrieve pending invite in the code. The work would be about
taking care of having a specialized method from either an existing or a new
TK service dedicated to this functionality.

Use this method to expose it via the API.
**** Request Org Access

- ref :: https://github.com/advthreat/iroh/issues/6271

We need to create another store with another Entity for access request to an Org.

#+begin_src clojure
(s/defschema OrgAccessRequest
  (st/merge
   {:id UUID
    :secret s/Str ;; should be generated via `iroh-auth.lib.security/password`
    :idp-mapping IdPMapping
    :user-email s/Str
    :org-id s/Str
    :status OrgAccessRequestStatus
    :created-at DateTime}
   (st/optional-keys
    {:user-name s/Str
     :user-nick s/Str
     :granted-role Role ;; the role granted if the request is accepted
     :approver-id UserId
     :approver-email UserEmail ;; email of the approver
     :updated-at DateTime
     })))
#+end_src

When a user request access to an organization.
We should create this object in DB.


#+begin_src
POST  /iroh/iroh-auth-ui/org-access/:org-id ;; request access to a matching org
#+end_src

#+begin_src http
POST /iroh/iroh-auth-ui/org-access/:org-id

Accept: application/json
Content-Type: application/json
User-Agent: ob-http
Authorization: Bearer ${user-identity-jwt}
#+end_src

With this, and if every check matches:

1. There is a known active admin of this org with an email with the same domain name
2. The org is active

We should:

1. create a new =OrgAccessRequest= object in DB.
2. Send emails (see the Email Notification of Org Request Accesses section)
**** Renew Org Access Request

There is an existing ~renew-org-access-request~ method from the
~OrgAcessRequestService~ that should be exposed.
Probably under the route ~/renew-org-access-request~.

In order to help the UI the return schema of the matching orgs should also
have a new optional boolean field ~can-renew?~.
So, only for :

1. matching org
2. with a pending org-access-request
3. such that this org access request could be renewed (the logic in the
   code say after 7 days from the last updated-at time)

You should dynamically add this field and set it to ~true~.

**** List Matching Orgs

- ref :: https://github.com/advthreat/iroh/issues/6273

#+begin_src http
GET  /iroh/iroh-auth-ui/matching-orgs
#+end_src

Given a ~UserIdentity~ with some email.
Should return every Org whose at least one admin has an email with the same
domain address.
So if you login via ~chuck@cisco.com~ you should return all orgs for which
there is at least one admin with a ~cisco.com~ email address.

Should return a list of objects with the following schema (sort to be defined):

#+begin_src clojure
{:org Org
 :org-nb-users s/Int
 :org-request-access-status OrgRequestAccessStatus}
#+end_src

*Note*

This endpoint is not straightforward.
We should list all current matching org, and concurrently list all current
=OrgRequestAccess= made by this =UserIdentity=.
We should also list all orgs for which there is already a pending invite.

We shall finally return a list of orgs sorted by the number of users (from
max to min) with the current status and remove the org for which there is
still a non expired pending invite.

If there is more than 6 orgs, we should only return
the firsts 6 of them but make it clear in the pagination headers that there
are more than 6 matching org so the frontend could react accordingly.
Another option would be to return a 400 with an error message.

***** CANCELED Hide some matching orgs manually

Optionally we might also retrieve the orgs marked as hidden for this =UserIdentity= in DB.

We should support query parameter to show all orgs, even the hidden ones so
a user should be able to unhide a previously hidden org.

**** Registration View

- ref :: https://github.com/advthreat/iroh/issues/6275

#+begin_src http
GET  /iroh/iroh-auth-ui/registration-view
#+end_src

Should return the same result as the union of the calls to
=matching-accounts=, =matching-orgs= and =pending-invites=.

#+begin_src clojure
{:matching-accounts [,,,]
 :matching-orgs [,,,]
 :pending-invites [,,,]}
#+end_src
**** CANCELED /OPTIONAL/ Hide Matching Org
:LOGBOOK:
- State "CANCELED"   from              [2022-03-21 Mon 10:53]
:END:

- ref :: https://github.com/advthreat/iroh/issues/6276

#+begin_src http
POST /iroh/iroh-auth-ui/hide-org/:org-id

Accept: application/json
Content-Type: application/json
User-Agent: ob-http
Authorization: Bearer ${user-identity-jwt}
#+end_src

Should save in the ~UserIdentity~ store a new ~org-id~ to hide.
This impacts the call to ~matching-orgs~ such that orgs marked as hidden
should not be displayed when calling the ~matching-org~ route.
**** DONE Redirects User

Create a new endpoint to redirect users using the value in the UserIdentity JWT.
See https://github.com/advthreat/iroh/issues/6280

#+begin_src http
GET /iroh/iroh-auth-ui/redirect
#+end_src

So this should be a route which should looks like this:

#+begin_src clojure
(GET "/redirect" []
  :summary "Given a code and some org-settings create a new account (new org and new user)"
  :description "This is an internal temporary route needed to select the user/org."
  :identity [session-redirect]
  (if (allowed-login-origin? session-redirect)
    (resp/found session-redirect)
    (err/bad-request :forbidden-redirect-url
                     (format "The redirect URL %s is not a valid redirect for login."
                             session-redirect)
                     {})))
#+end_src

*** =1= /OPTIONAL/ Sync UserIdentity on all successful SecureX login
/Estimate: 1 rcd/

- ref :: https://github.com/advthreat/iroh/issues/6367

From #6076 we have a new CRUD service that should keep track of `UserIdentity`.
Mainly this information could be used as providing more details than what is currently set int he user's `idp-mapping`. Typically we have access to the email of the UserIdentity, user name from the IdP, and probably a lot more details that will be useful.

So every time a successful login occurs we should update the UserIdentity object last logged in date (as we do for users) which will probably be quite useful.

** =3= Email Notification of Org Request Accesses
/Estimate: 3 rcd after email+html templates/
- ref :: https://github.com/advthreat/iroh/issues/6368

- List all the admins of the requested org.
2.a. If there is fewer admins than a number that could be configured in the
     node configuration. Then we send an email to all admins.
2.b. If there are more admins than this specific number, then we randomly
     chose this maximal number of admins and send them an email notification.

*TODO* Have an email template (both HTML and Text)
*** Notes

Need to be able to trigger a new request to join after 7 days.

** =2= Org Requests CRUD API for Admins of the Orgs
/Estimate: 2 rcd after mail templates + text/
- ref :: https://github.com/advthreat/iroh/issues/6369

There should be a CRUD API restricted to the ~admin/user-mgmt/org-requests~ scope:

- ~GET  /iroh/user-mgmt/org-requests~ list pending org access requests
- ~GET  /iroh/user-mgmt/org-requests/<id>~ read a single org access request
- ~PATCH /iroh/user-mgmt/org-requests/<id>~ Grant or Reject the access

*** List

~GET  /iroh/user-mgmt/org-requests~

If no parameter is provided, only list pending =OrgAccessRequests= of the org
of the caller.
Otherwise we could pass the query-parameter =status= with the following
value(s):

- =pending=
- =accepted=
- =rejected=

Note we should probably support duplicate statuses.
Ex:

~GET /iroh/user-mgmt/org-requests?status=accepted&status=pending~

*** Read

~GET /iroh/user-mgmt/org-requests/org-request-id~

Should returns a 404 if not found or the single Org Access Request object.

*** Patch the Org Access

~PATCH /iroh/user-mgmt/org-requests/<id>~ Grant/Reject the access

The body should be a JSON Object with:
- an optional field =role= whose value could be either =admin= or =user= (default
  to =user=)
- a mandatory =status= field whose value could be either =accepted= or =rejected=.

A few examples of valid values:

#+begin_src js
{"granted-role":"admin"
 "status":"accepted"}
#+end_src

#+begin_src js
{"status":"accepted"}
#+end_src

#+begin_src js
{"status":"rejected"}
#+end_src

During the call we should:

1. Create a new user with:

#+begin_src clojure
{:user-id (gen-uuid)
 :org-id (:org-id org-access-request)
 :user-email (:user-email org-access-request)
 :idp-mappings [(:idp-mapping org-access-request)]
 :user-name (:user-name org-access-request)
 :user-nick (:user-nick org-access-request)
 :role (get-in request [:body :role])
 :enabled? true
 }
#+end_src

2. Send an email to user confirming his access was granted.

*TODO* have an email template + text.
** =3= /OPTIONAL/ Email Templating Service & Interface

We should create new endpoints for the UI team to be able to upload new
Email templates.
We currently have a mail-templates mechanism only for invites.
We should have a similar system for all the email sent in our IROH-Auth
system.

With the Org Access Request Service, we will now have 3 new different kinds
of emails:

- the email sent to the admins of an org when a user request access to some org
- the email sent to the user when an admin accept the org request
- the email sent to the user when an admin reject the org request

In order to achieve this; you should:

1. create a new ~MailTemplateService~ which contains two parts:
   a. a simple layer on top of ~iroh-crud~ but for ~MailTemplate~ (look at the code for the
   correct schema) BUT any update on any template should be kept in some
   history so we could undo any change and we should keep track of the
   author of the changes.
   a. expose an helper function to send emails with the following API:
      ~(send-templated-email [template-id template-params])~
2. create a new endpoint in ~iroh-admin~ so user with the correct privileges
   could update the templates. Use the scope ~iroh-master/dev-ui~ for the routes.
3. In the ~OrgAccessRequestService~ use this ~MailTemplateService~ to send the
   email to the users. This refacto should potentially be used for the
   Invite mechanism, but as it appear to work fine today we might decide
   not to touch it.

*** Dev details

Here are some dev details that could probably be copy/pasted as-is:

#+begin_src clojure
(s/defschema MailTemplate
  (st/optional-keys
   {:subject s/Str
    :subtitle s/Str
    :text s/Str ;; could contain HTML
    :mail-source s/Str}))

(defprotocol EMailTemplatingService
  ;; CRUD operations
  (create-email-template [this email-template-id new-email-template])
  (get-email-template [this email-template-id])
  (delete-email-template [this email-template-id])
  (update-email-template [this email-template-id email-template])
  (patch-email-template [this email-template-id email-template-patch])
  ;; History
  (search-email-template-history
    [this email-template-id filter-map text-match pagination-params]
    "given an email-template-id, return the list of matched template-id saved in history.")
  (revert-email-template [this email-template-id email-template-history-id]
    "given a template-id and and corresponding email-template-history-id
     perform an update to revert the change to this old template")
  (undo-email-template [this email-template-id]
    "revert-email-template on the previous one.
     Beware reverting/undoing is also saved in history.")

  ;; SendEmails
  (send-templated-email [this email-template-id template-params]
    "send an email using some template id and template-params
     are object that will be used by the mustache templating system to fill the template."))

(defservice email-templating-service
  EmailTemplatingService
  [EMailService]
  ,,,)
#+end_src

*SECURITY NOTES:*
If the DB is empty, there should always be a "default template by
template-id" in the code.

As we are giving some power out of the backend regarding some security
concerns.
We should only provide the mandatory fields to the template params and take
care of removing the other ones.
Typically, the org-access-request secret should be dissoc'd and the admin
user-id should be provided unencrypted for the correct emails, etc…

* Notes

** Number of user with a matching domain email by domain

#+begin_src
csvtool namedcol "user-email" users.csv |sed 's/[^@]*@//'|sort|uniq -c|sort -n
|tail
    134 scitum.com.mx
    147 purdue.edu
    186 atp-us.com
    201 sentinel.com
    253 cognizant.com
    337 port53tech.com
    842 dcloud.cisco.com
   1315 gmail.com
   2036 unknown mail
  14762 cisco.com
#+end_src

The maximal number of user by email domain which are not Cisco 337 users.
And >= 134 only for 10 on >15k domains.

** Number of orgs with more than 100 users

Here is the list of the total number of orgs with >100 users in prod NAM:

#+begin_src
101 a9876101-3d6a-4a6b-b09a-175d554e446a
107 18fc4fd9-fb31-48bd-a828-a8a0783a1d2e
115 threatgrid:7
118 threatgrid:1
136 75016e07-5850-4054-a537-eea5d2897d17
144 b3c172f8-792a-4c0e-b572-b580e1a8c6b6
144 cb3958cf-992d-44f6-8803-de684b614f97
146 9755f0e4-57b9-4514-a8c6-93d66edf5f9b
146 ce6c4a4b-fd10-4a07-a23b-88e5f1580565
153 e1ee0f83-56d2-4dd8-8463-dc6bfd6afcb8
195 9ec78968-adff-40c2-b57c-7bc2ed2132cc
204 681c3549-1c52-4ca3-b88c-3de68438bb03
220 a94805d9-ad26-47f6-9fa4-647cbd89cb6c
239 7045f4c2-3a38-49ff-9bb5-01052219e8da
267 6fcb8a6f-c043-4842-a1a4-9bfff5f57f15
347 56e851ae-8146-4834-9257-d0068d5885ba
352 7df4fb52-3ded-4e75-9160-eaa07614192a
381 a1ff6ab0-6693-4c2d-a203-54db28d34e78
472 f81d494c-ec2d-4589-80b2-1941a3a9d875
473 399a1a44-f230-4997-8195-f111172a503e
560 f86053fc-f173-488f-9d55-825fc0117239
694 threatgrid:1182
#+end_src

So 22 orgs out of 29541 orgs.

** Number of orgs by email domain name

Matching domain names:

#+begin_src
  103 yahoo.com
  115 anm.com
  116 corebts.com
  118 hotmail.com
  124 ubc.ca
  132 sfsnort.com
  134 scitum.com.mx
  147 purdue.edu
  186 atp-us.com
  201 sentinel.com
  253 cognizant.com
  337 port53tech.com
  842 dcloud.cisco.com
 1315 gmail.com
 2036 unknown mail
14762 cisco.com
#+end_src

** Number of matching orgs per email domain name

So 16 domain email with >100 users while there are 15542 different email domains in the DB in PROD NAM.

The number of matching orgs per email filtered by orgs with >6 matching orgs:

#+begin_src
      7 businessinformationgroup.com
      7 carouselindustries.com
      7 censystems.com.mx
      7 chickasawtel.com
      7 choctawnation.com
      7 deloitte.com
      7 hammer.net
      7 ikusi.com
      7 its-networks.com
      7 live.com
      7 mac.com
      7 mail.mil
      7 mfec.co.th
      7 ndc.com.br
      7 orange.com
      7 orben.com
      7 redesmayab.com
      7 securesoftcorp.com
      7 spcinternacional.com
      7 teledinamica.com.mx
      7 tetradefense.com
      7 us.logicalis.com
      8 att.com
      8 compucom.com
      8 comstor.com
      8 dsitech.com
      8 gdt.com
      8 getgds.com
      8 ineteng.com
      8 mitgs.com
      8 outcomex.com.au
      8 telcion.com
      8 zebra.com
      9 apple.com
      9 aqueducttech.com
      9 conscia.com
      9 icloud.com
      9 katalystng.com
      9 protonmail.com
      9 securview.com
      9 softserveinc.com
      9 techdata.com
      9 telefonica.com
      9 tfmx.com
      9 veytec.com
     10 aspiretransforms.com
     10 comstor-la.com
     10 me.com
     10 trace3.com
     10 uncommonx.com
     11 3rtnetworks.com
     11 axity.com
     11 cognizant.com
     11 compunet.biz
     11 dimensiondata.com
     11 hyetechnetworks.com
     11 italtel.com
     11 meraki.net
     11 sonda.com
     11 state.nm.us
     11 tgs-mtc.com
     11 voseda.com
     11 wwt.com
     12 5thcolumn.net
     12 bvstv.com
     12 ceriumnetworks.com
     12 iconvergence.com
     12 ignitessg.com
     12 marconet.com
     12 promenet.com
     13 atsg.net
     13 hbs.net
     13 oneneck.com
     14 atp-us.com
     14 scitum.com.mx
     14 teletex.com.br
     15 un-mx.net
     16 thinkgard.com
     17 digitalhands.com
     17 ingrammicro.com
     18 arrayasolutions.com
     18 ccd.com.co
     19 la.logicalis.com
     20 eplus.com
     20 global.ntt
     22 connext.com.mx
     22 sourcefire.com
     23 anm.com
     23 presidio.com
     26 cdw.com
     26 nwnit.com
     30 outlook.com
     30 sentinel.com
     33 insight.com
     37 convergeone.com
     46 port53tech.com
     51 corebts.com
     51 hotmail.com
     59 yahoo.com
     66 sfsnort.com
     76 external.cisco.com
    188 dcloud.cisco.com
    199 unknown mail
    577 gmail.com
   4817 cisco.com
#+end_src


22 domains email will match 6 orgs

#+begin_src
6 2s.com.br
6 bynet.co.il
6 charter.ca
6 citynet.net
6 comcast.net
6 comsoltx.com
6 evcon-group.com
6 fxti.ca
6 gbm.net
6 hainc.com
6 ibm.com
6 iementor.com
6 infranetgroup.com
6 lannet.net
6 netconsulting.com.br
6 netsync.com
6 opendns.com
6 shi.com
6 softchoice.com
6 solucionesorion.com
6 synnapex.com
6 synnex.com
6 tcs.com
6 teltecsolutions.com.br
6 ubc.ca
6 yssy.com.br
#+end_src

- 22 domains email will match 6 orgs
- 42 domains email will match 5 orgs
- 90 domain emails will match 4 orgs.
- 339 domain emails will match 3 orgs.
- 1606 domain emails will match 2 orgs.
- 8591 domain emails will match a single org.

** Number of admins per org

On 20816 activated orgs:

- 20365 (97.8%) have less than 10 admins.
- 451 (2.2%) have more than 10 admins
- 167 (0.8%) have more than 15 admins
- 89  (0.4%) have more than 20 admins

#+begin_src
     21 1850c06a-e8a5-44d4-a9a3-9410ab278454,evansville.in.gov
     21 a684a119-0c88-458e-8291-a890998be3c6,cisco.com
     21 a72a2650-ab6b-4c81-a2cf-dc15be3025f9,cisco.com
     21 ac44b1c0-2f6a-4536-9d03-0b717c1c843c,dconc.gov
     21 cb3770c1-64c5-4925-83f5-290fadc355b6,adexus.cl
     21 d45dc5fd-c336-45a3-848f-b28bc22e122c,cisco.com
     21 fe331828-8f82-4371-91e7-257c7745645c,arrayasolutions.com
     22 2465d67c-bb5e-4ad1-8b0e-a67b7ce36569,network-data.com
     22 3620d167-1aeb-4dca-b7f7-dc66acfe9c08,airnz.co.nz
     22 67eea099-b310-47e9-8036-ae44bfe30e0b,cisco.com
     22 713fc255-f061-4557-9fda-aec7f385dd9f,sentinel.com
     22 7cf1057a-ed3c-4da7-a251-22e106691acd,atp-us.com
     23 04b4d7b7-542c-47e4-8c45-2cde5dde09f8,uncp.edu
     23 67bcaf0d-3eef-49ac-b15e-ef5da3c3472f,cisco.com
     23 70d84b1c-a2f1-403b-bf90-af6ffdec4c84,sentinel.com
     23 eb388261-c20e-48a9-9554-8816b63d182c,cisco.com
     24 42ac91b9-d9df-48d8-a1d0-8c76988d0a9f,cisco.com
     24 684df03f-e062-4696-8271-5ac1ce00ac94,cisco.com
     24 71295ced-1fd3-44bd-926f-2f23712cf2a9,egyde.ca
     24 90ab7d03-2507-4700-b07e-ddfcdc51e652,nm.org
     25 086ab5fd-1b56-4abb-b652-c47a3d8eab0c,digitalhands.com
     25 23058a37-2181-4dee-8057-57722a6efb0e,rsfh.com
     25 328bc4ec-b74e-4db1-b7ef-4c391ed270c9,aspiretransforms.com
     25 491257d7-6257-48e4-a103-1fd31b01daf5,cityoftulsa.org
     25 5934ea5e-0c08-4d37-8d36-c4a1fe3dbb01,cisco.com
     25 6f6027d8-3348-4f06-8e85-48bb1215c9d8,aqueducttech.com
     25 81318155-45ca-49ff-9328-18ce53e97547,sentinel.com
     25 ac782c34-66ad-4b26-a245-6943e6d7146a,illinois.gov
     25 c3ffe79f-77d9-433c-b76b-dda8b776fb8c,telconet.ec
     25 f81d494c-ec2d-4589-80b2-1941a3a9d875,cisco.com
     26 39f7ec21-203f-4817-8737-bb889457495c,metlife.com
     26 b8bf2598-2836-4156-ad5c-14bdedd3acf0,abm.com
     26 cfef1746-df05-45a2-b421-bc2cdb68b233,cdk.com
     26 threatgrid:1,cisco.com
     26 threatgrid:9647,cisco.com
     27 572df3d2-d561-46eb-95df-d85479f5f98f,wisc.edu
     27 a2d41370-0db2-4605-8a1b-6bbf2041f0de,alvarezandmarsal.com
     27 e2cd17b2-4fd7-439c-a414-62a3737abe96,cisco.com
     28 2a8fecff-3c72-4177-97d6-39d6eb1338a6,choa.org
     28 5643fbcc-453f-4ea6-a4bd-36414d10b64b,myoccu.org
     28 8b78a2fd-1995-4ba7-a4fd-75cfefafb7a3,cisco.com
     29 95e571e0-d496-46ae-a85f-fad859a1f97a,txdot.gov
     29 b376744b-cf4a-4434-a43d-5d9304d01d03,altagas.ca
     29 b71d47b8-3d53-44af-bbcc-5a7bbf5de4ec,cisco.com
     30 1bb4024c-0d38-448e-9ec0-32fb17b3c881,ferc.gov
     31 08f89d0d-d3a3-43cb-9605-dfe30109d71f,verint.com
     31 2f878d5d-0212-4948-9dbd-18ae3afdced9,cisco.com
     32 b230c988-2d9a-47bb-9404-6b7003d238ea,harley-davidson.com
     33 36c4891e-fbf2-4e55-9e73-47320d5b8293,scitum.com.mx
     33 77c16812-c601-4a7a-87f8-c60c0d354277,ahs.ca
     33 97ac3ac5-604d-4284-8f84-a3c4d4db5f1f,sanfordhealth.org
     33 cd2da7c0-db7e-4446-ab69-f8e6097e92d6,messa.org
     34 1088e378-b2b3-4ab5-bc68-128885662140,ntrs.com
     34 threatgrid:299019,cisco.com
     36 e5745b0f-8ad7-4290-b008-16cb4c40b026,scitum.com.mx
     36 f598e1b3-db04-42a7-ae17-31ffde6aa94f,cisco.com
     37 1c9cb3e5-d876-4e4f-853f-f09ee5ea2f3c,cisco.com
     37 b3c172f8-792a-4c0e-b572-b580e1a8c6b6,west.com
     38 1d46f11e-17ab-4df6-8c8b-8dafd1dbde63,cisco.com
     38 33ddd5e5-d8ea-40b8-9c46-2f25eefc983b,demandscience.com
     38 8b8ae50d-0da7-4461-94e4-3d5a8d23a791,cisco.com
     39 2c638d3a-57be-49fb-8896-c5a1c4cb402f,cisco.com
     40 a127256e-d154-4535-9824-10f24e938ece,evopayments.com
     40 db6fa2c3-3271-4ee6-8dae-47d21eb0f59c,cisco.com
     41 4f5f7fb1-3e51-4808-99fb-02a9d3d12d15,dcloud.cisco.com
     41 c55ed22f-4204-424d-9f7e-c71470b97c73,libertyutilities.com
     42 1841d0d7-36df-45e7-8c94-43c7710b80cb,cisco.com
     42 4ca63e42-4a71-4c85-9a28-06afac0c2273,cisco.com
     44 ea3f7333-aede-44b2-b3d5-0a9e0e1b52a2,cisco.com
     47 d76c035d-e896-438c-8d75-158be85fc958,cisco.com
     50 473db1e7-cc1e-434f-b33f-642abf4b1d2d,cisco.com
     51 77145ba3-ccb7-40ec-b91f-b869cdcf2c8b,cisco.com
     55 threatgrid:1182,cisco.com
     64 7045f4c2-3a38-49ff-9bb5-01052219e8da,cognizant.com
     66 f47a89bf-5d2e-4392-b770-ad4821a82acf,cisco.com
     68 18fc4fd9-fb31-48bd-a828-a8a0783a1d2e,cisco.com
     88 6fcb8a6f-c043-4842-a1a4-9bfff5f57f15,cisco.com
     91 a9876101-3d6a-4a6b-b09a-175d554e446a,shurtape.com
    116 75016e07-5850-4054-a537-eea5d2897d17,cisco.com
    142 ce6c4a4b-fd10-4a07-a23b-88e5f1580565,cisco.com
    193 9ec78968-adff-40c2-b57c-7bc2ed2132cc,cisco.com
    204 681c3549-1c52-4ca3-b88c-3de68438bb03,cisco.com
    372 399a1a44-f230-4997-8195-f111172a503e,cisco.com
    560 f86053fc-f173-488f-9d55-825fc0117239,cisco.com
#+end_src