deft/reports/FY24Q2-tmp-report.org

#+title: FY24Q2 Report
#+subtitle: logs goes 4 months back
#+date: 2024-01-26
#+options: H:6 ^:nil
* IROH
** lead


***  [1]

**** iroh [1]


_between 3 and 4 months old_

- Don't fire disabled webhooks [[https://github.com/advthreat/iroh/pull/8741][#8741]]
** data


*** Mario Aquino [15]

**** iroh [10]

- Fix disabled threat-hunt test [[https://github.com/advthreat/iroh/pull/8814][#8814]]
- Update incident_time when updating incident status [[https://github.com/advthreat/iroh/pull/8801][#8801]]
- incident enrichment activity diagram [[https://github.com/advthreat/iroh/pull/8712][#8712]]
- Separate Risk score & incident enrichment [[https://github.com/advthreat/iroh/pull/8751][#8751]]

_between 3 and 4 months old_

- Improve safe-filtering [[https://github.com/advthreat/iroh/pull/8731][#8731]]
- iroh-async: Flatten Datadog context [[https://github.com/advthreat/iroh/pull/8706][#8706]]
- iroh-async logging & tracing context [[https://github.com/advthreat/iroh/pull/8705][#8705]]
- Socket timeout milliseconds (not seconds) [[https://github.com/advthreat/iroh/pull/8690][#8690]]
- Risk Score socket-timeout [[https://github.com/advthreat/iroh/pull/8687][#8687]]
- Threat Hunt Module Exclusion [[https://github.com/advthreat/iroh/pull/8646][#8646]]
**** tenzin-config [5]

- Increase conn-manager thread count after PROD performance monitoring [[https://github.com/advthreat/tenzin-config/pull/1042][#1042]]
- Increase thread pool size for EU private intel conn mgr [[https://github.com/advthreat/tenzin-config/pull/1039][#1039]]
- Increase connection mgr thread pool for NAM/EU/TEST [[https://github.com/advthreat/tenzin-config/pull/1030][#1030]]

_between 3 and 4 months old_

- Increase thread pool size for engine connection manager [[https://github.com/advthreat/tenzin-config/pull/1012][#1012]]
- Config for skipping Private Intel during investigation threat hunt [[https://github.com/advthreat/tenzin-config/pull/1009][#1009]]

***  [3]

**** ctia [1]


_between 3 and 4 months old_

- fix wait_for for delete search [[https://github.com/threatgrid/ctia/pull/1399][#1399]]
**** iroh [2]


_between 3 and 4 months old_

- bundle import activity diagrams [[https://github.com/advthreat/iroh/pull/8708][#8708]]
- scoring at bundle import [[https://github.com/advthreat/iroh/pull/8694][#8694]]

*** Ambrose Bonnaire-Sergeant [16]

**** ctia [10]

- Fix 2XX response swagger/coercion, ban ~:return~ [[https://github.com/threatgrid/ctia/pull/1407][#1407]]
- Remove asset properties/mapping merging during bundle patch [[https://github.com/threatgrid/ctia/pull/1408][#1408]]

_between 3 and 4 months old_

- Fix :body descriptions [[https://github.com/threatgrid/ctia/pull/1409][#1409]]
- Fix POST /bulk schema checking [[https://github.com/threatgrid/ctia/pull/1406][#1406]]
- Use prn instead of pprint for logs [[https://github.com/threatgrid/ctia/pull/1401][#1401]]
- Eval routes and options given to ~context~ at initialization time [[https://github.com/threatgrid/ctia/pull/1394][#1394]]
- Use ~st/merge~ to merge schemas instead of ~into~ [[https://github.com/threatgrid/ctia/pull/1398][#1398]]
- Never match existing asset-* entities when patch-existing=false [[https://github.com/threatgrid/ctia/pull/1395][#1395]]
- Re-enable incident tests [[https://github.com/threatgrid/ctia/pull/1393][#1393]]
- Add external_ids to investigation select fields [[https://github.com/threatgrid/ctia/pull/1392][#1392]]
**** iroh [4]

- Generate valid DI auth tokens for incident subscriptions [[https://github.com/advthreat/iroh/pull/8804][#8804]]
- Fix ~(reset)~ [[https://github.com/advthreat/iroh/pull/8799][#8799]]
- Subscribe to incident asset rescoring via DI [[https://github.com/advthreat/iroh/pull/8699][#8699]]

_between 3 and 4 months old_

- Only pass default patch-existing query params in bundle/import proxy if patch-existing=true [[https://github.com/advthreat/iroh/pull/8725][#8725]]
**** tenzin-config [2]

- Add config for DI client in iroh-engine [[https://github.com/advthreat/tenzin-config/pull/1036][#1036]]
- Add device insights url to iroh-engine config [[https://github.com/advthreat/tenzin-config/pull/1014][#1014]]
** integrations


***  [4]

**** iroh [4]


_between 3 and 4 months old_

- JMX metrics for clj-http connection manager [[https://github.com/advthreat/iroh/pull/8765][#8765]]
- Always decompress the body when status is not 2xx [[https://github.com/advthreat/iroh/pull/8527][#8527]]
- Restore default expiration (24h) for the local session token [[https://github.com/advthreat/iroh/pull/8747][#8747]]
- Workflow event schema changes [[https://github.com/advthreat/iroh/pull/8656][#8656]]

*** Kirill Chernyshov [10]

**** iroh [7]

- Refactor data streams service [[https://github.com/advthreat/iroh/pull/8793][#8793]]
- DRY'ed out client-creds-token namespace [[https://github.com/advthreat/iroh/pull/8783][#8783]]
- Kafka connect monitoring [[https://github.com/advthreat/iroh/pull/8278][#8278]]

_between 3 and 4 months old_

- Handle undelivered records [[https://github.com/advthreat/iroh/pull/8634][#8634]]
- Events data retention enforcement job [[https://github.com/advthreat/iroh/pull/8722][#8722]]
- Iroh events data retention implementation [[https://github.com/advthreat/iroh/pull/8666][#8666]]
- Iroh events postgres data retention [[https://github.com/advthreat/iroh/pull/8693][#8693]]
**** tenzin-config [3]

- Add ES sink connector v2 to test full migration [[https://github.com/advthreat/tenzin-config/pull/1035][#1035]]
- Fix broken data stream on TEST [[https://github.com/advthreat/tenzin-config/pull/1034][#1034]]
- Config for DataStreams service [[https://github.com/advthreat/tenzin-config/pull/1033][#1033]]

*** Shafiq [7]

**** iroh [4]

- Fix schema of proxy health check [[https://github.com/advthreat/iroh/pull/8827][#8827]]
- Add string matching for health check [[https://github.com/advthreat/iroh/pull/8815][#8815]]
- Fallback to iroh-events store when kafka send fails [[https://github.com/advthreat/iroh/pull/8786][#8786]]

_between 3 and 4 months old_

- Fix mapping for incident events [[https://github.com/advthreat/iroh/pull/8703][#8703]]
**** tenzin-config [3]


_between 3 and 4 months old_

- Swtich to new sink-connector in INT [[https://github.com/advthreat/tenzin-config/pull/1024][#1024]]
- Add updated sink connector for all envs [[https://github.com/advthreat/tenzin-config/pull/1021][#1021]]
- Add sink connector for new iroh-event datastream [[https://github.com/advthreat/tenzin-config/pull/1019][#1019]]
** auth


*** bartuka [26]

**** iroh [22]

- [IROH Auth] Support FMC in the ~jwks~ service [[https://github.com/advthreat/iroh/pull/8830][#8830]]
- [IROH Auth] Fix DI onboarding in Universal Provisioning Flow [[https://github.com/advthreat/iroh/pull/8813][#8813]]
- Revert "[IROH Auth] support for FMC token in JWKS Service" [[https://github.com/advthreat/iroh/pull/8816][#8816]]
- [IROH Auth] support for FMC token in JWKS Service [[https://github.com/advthreat/iroh/pull/8808][#8808]]
- [IROH Auth] Check QA ~callback_url~ to complete provisioning tests [[https://github.com/advthreat/iroh/pull/8763][#8763]]

_between 3 and 4 months old_

- [IROH Auth] better swagger descriptions for Universal Provisioning [[https://github.com/advthreat/iroh/pull/8752][#8752]]
- [IROH Auth] remove empty strings from ~client-id~ got from Vault [[https://github.com/advthreat/iroh/pull/8760][#8760]]
- [IROH Auth] add missing scope to get OKTA JWT [[https://github.com/advthreat/iroh/pull/8759][#8759]]
- [IROH Auth] add logs to investigate get okta jwt [[https://github.com/advthreat/iroh/pull/8758][#8758]]
- [IROH Auth] bugfix - ~client/post~ should use ~:form-params~ instead of ~:body~ [[https://github.com/advthreat/iroh/pull/8753][#8753]]
- [IROH Auth] bugfixes - arity exception, change ~product-response~ datatype, change ~pmap~ to ~map~ [[https://github.com/advthreat/iroh/pull/8738][#8738]]
- [IROH Auth] bugfix - fix urls in ~POST /tenants~ returned value and payload field names [[https://github.com/advthreat/iroh/pull/8733][#8733]]
- [IROH Auth] Cache OKTA JWT used for provisioning callback [[https://github.com/advthreat/iroh/pull/8727][#8727]]
- [IROH Auth] Improvements on logs and error handling to UniversalProvisioning and JWKSService [[https://github.com/advthreat/iroh/pull/8707][#8707]]
- [IROH Auth] Duplicate ~universal-provisioning~ web routes to accept IROH JWTs [[https://github.com/advthreat/iroh/pull/8675][#8675]]
- [IROH Auth] Expose ~callbacks packages~ store to check Universal Provisioning status [[https://github.com/advthreat/iroh/pull/8702][#8702]]
- [IROH Auth] Improve 202 Accepted response for ~/universal-provisioning/create-tenants~ [[https://github.com/advthreat/iroh/pull/8701][#8701]]
- [IROH Auth] Fix name convention to callbacks route in  Universal Provisioning flow [[https://github.com/advthreat/iroh/pull/8691][#8691]]
- [IROH Auth] Gen OKTA JWT to callback request in Universal Provisioning flow [[https://github.com/advthreat/iroh/pull/8673][#8673]]
- bugfix - missing ~UniversalProvisioningCallbackService~ to deploy IROH nodes [[https://github.com/advthreat/iroh/pull/8680][#8680]]
- [IROH Auth] Check pending provisions from time-to-time. Endpoint that will be called by OPS tick [[https://github.com/advthreat/iroh/pull/8674][#8674]]
- [IROH Auth] Add callback handler to receive provisioning status from downstream apps [[https://github.com/advthreat/iroh/pull/8633][#8633]]
**** tenzin-config [4]

- FMC base-urls to configure JWKS [[https://github.com/advthreat/tenzin-config/pull/1040][#1040]]

_between 3 and 4 months old_

- [IROH Auth] bugfix - add config to okta jwks [[https://github.com/advthreat/tenzin-config/pull/1017][#1017]]
- [IROH Auth] add OKTA config for Universal Provisioning flow #1010 [[https://github.com/advthreat/tenzin-config/pull/1013][#1013]]
- add universal_provisioning_callbacks store [[https://github.com/advthreat/tenzin-config/pull/1011][#1011]]

*** Yann Esposito [27]

**** iroh [17]

- Fix PIAM Universal Provisioning routes [[https://github.com/advthreat/iroh/pull/8828][#8828]]
- Should fix open impersonate flaky test [[https://github.com/advthreat/iroh/pull/8809][#8809]]
- Keep track of impersonators [[https://github.com/advthreat/iroh/pull/8736][#8736]]
- Restrict TAC routes to admins [[https://github.com/advthreat/iroh/pull/8794][#8794]]

_between 3 and 4 months old_

- Remove with-tk  [[https://github.com/advthreat/iroh/pull/8779][#8779]]
- Code/Test Improvements [[https://github.com/advthreat/iroh/pull/8767][#8767]]
- add a test for matching schema [[https://github.com/advthreat/iroh/pull/8770][#8770]]
- Custom Role Design doc [[https://github.com/advthreat/iroh/pull/8497][#8497]]
- Attempt to improve error message of match? [[https://github.com/advthreat/iroh/pull/8769][#8769]]
- Use ~cid~ for ~trace_id~ when present [[https://github.com/advthreat/iroh/pull/8754][#8754]]
- Support public client for custom routes [[https://github.com/advthreat/iroh/pull/8749][#8749]]
- Add playbook scope [[https://github.com/advthreat/iroh/pull/8739][#8739]]
- Fix webhook race condition risk [[https://github.com/advthreat/iroh/pull/8728][#8728]]
- Call get-org only once for org-virtual user [[https://github.com/advthreat/iroh/pull/8724][#8724]]
- Use a cache for entitlement summaries [[https://github.com/advthreat/iroh/pull/8667][#8667]]
- upgrade jetty version [[https://github.com/advthreat/iroh/pull/8714][#8714]]
- Remove a forgotten pretty printer [[https://github.com/advthreat/iroh/pull/8713][#8713]]
**** iroh-scripts [6]

- save improvements

_between 3 and 4 months old_

- create dashboard clients
- Support client aliases in get-client
- Update client with client-aliases
- many new scripts
- added a 1-time script
**** tenzin-config [2]


_between 3 and 4 months old_

- Check vault templating error [[https://github.com/advthreat/tenzin-config/pull/1023][#1023]]
- Add Universal Provisioning Services [[https://github.com/advthreat/tenzin-config/pull/1015][#1015]]
**** xdr-provisioning [2]


_between 3 and 4 months old_

- Manage entitlements add-ons
- add a re-provisioning script that rerun onboardings

*** Olivier Barbeau [25]

**** iroh [15]

- Check the list of services for a node type [[https://github.com/advthreat/iroh/pull/8800][#8800]]
- Fix merge error on PR 8784 [[https://github.com/advthreat/iroh/pull/8797][#8797]]
- [IROH configuration]: Move role-web-service config to default tk files [[https://github.com/advthreat/iroh/pull/8782][#8782]]
- [IROH configuration]: Universal Provisioning Services config refactor [[https://github.com/advthreat/iroh/pull/8784][#8784]]
- [IROH configuration]: explicit name for generated conf and meta [[https://github.com/advthreat/iroh/pull/8785][#8785]]
- Clean bootstrap.cfg; remove tmp file [[https://github.com/advthreat/iroh/pull/8781][#8781]]

_between 3 and 4 months old_

- Add few additional tests to iroh services [[https://github.com/advthreat/iroh/pull/8762][#8762]]
- rewrite tests [[https://github.com/advthreat/iroh/pull/8773][#8773]]
- more info for debugging [[https://github.com/advthreat/iroh/pull/8717][#8717]]
- [IROH configuration]: general documentation [[https://github.com/advthreat/iroh/pull/8764][#8764]]
- update developer doc for api-gateway [[https://github.com/advthreat/iroh/pull/8723][#8723]]
- Some test clean-up [[https://github.com/advthreat/iroh/pull/8716][#8716]]
- High volume of SQL queries for a single observe/deliberate call [[https://github.com/advthreat/iroh/pull/8682][#8682]]
- Remove the state of module instances in ~obfuscate-module-instance~ [[https://github.com/advthreat/iroh/pull/8670][#8670]]
- E8388: update proxy-endpoints-metadata endpoint and metadata [[https://github.com/advthreat/iroh/pull/8663][#8663]]
**** tenzin-config [10]

- Deep merge for vectors and sets with duplicates check [[https://github.com/advthreat/tenzin-config/pull/1032][#1032]]
- Reduce configuration duplicates - config.edn part [[https://github.com/advthreat/tenzin-config/pull/1031][#1031]]
- Reduce configuration duplicates - bootstrap.cfg part [[https://github.com/advthreat/tenzin-config/pull/1028][#1028]]
- Move role-web-service config to IROH [[https://github.com/advthreat/tenzin-config/pull/1026][#1026]]
- Move Universal Provisioning Services config to IROH [[https://github.com/advthreat/tenzin-config/pull/1027][#1027]]
- Clean bootstrap cfg [[https://github.com/advthreat/tenzin-config/pull/1025][#1025]]

_between 3 and 4 months old_

- Remove the ~:registration~ flag in all environments [[https://github.com/advthreat/tenzin-config/pull/963][#963]]
- Remove the ~:merge-users-by-email~ flag in all environments [[https://github.com/advthreat/tenzin-config/pull/962][#962]]
- Remove the ~:account-activation-optim~ flag in all environments as it is now activated everywhere. [[https://github.com/advthreat/tenzin-config/pull/961][#961]]
- Remove the ~xdr-roles~ flag in all environments [[https://github.com/advthreat/tenzin-config/pull/964][#964]]

*** (Yogsototh) [8]

**** iroh-scripts [6]

- save improvements

_between 3 and 4 months old_

- create dashboard clients
- Support client aliases in get-client
- Update client with client-aliases
- many new scripts
- added a 1-time script
**** xdr-provisioning [2]


_between 3 and 4 months old_

- Manage entitlements add-ons
- add a re-provisioning script that rerun onboardings
** iroh-ops


***  [0]


***  [0]

* Other
** Other


*** II [4]

**** iroh [4]

- Merges module type props on create and update health check [[https://github.com/advthreat/iroh/pull/8845][#8845]]
- Implements v2 threat hunting [[https://github.com/advthreat/iroh/pull/8833][#8833]]
- This should fix issue with parent settings not used on create-patch [[https://github.com/advthreat/iroh/pull/8822][#8822]]

_between 3 and 4 months old_

- Adds insights scope to allowed Automation scopes in gen-ao-jwt [[https://github.com/advthreat/iroh/pull/8678][#8678]]

***  [4]

**** iroh [4]

- Merges module type props on create and update health check [[https://github.com/advthreat/iroh/pull/8845][#8845]]
- Implements v2 threat hunting [[https://github.com/advthreat/iroh/pull/8833][#8833]]
- This should fix issue with parent settings not used on create-patch [[https://github.com/advthreat/iroh/pull/8822][#8822]]

_between 3 and 4 months old_

- Adds insights scope to allowed Automation scopes in gen-ao-jwt [[https://github.com/advthreat/iroh/pull/8678][#8678]]

*** Scott McLeod [1]

**** iroh [1]

- tk store: Add delete-search method #8213 [[https://github.com/advthreat/iroh/pull/8692][#8692]]

*** Brooke Swanson [3]

**** ctia [1]


_between 3 and 4 months old_

- Add brookeswanson to codeowners. [[https://github.com/threatgrid/ctia/pull/1396][#1396]]
**** iroh [1]


_between 3 and 4 months old_

- Add no-doc true and prevent explosion due to mismatched types. [[https://github.com/advthreat/iroh/pull/8548][#8548]]
**** tenzin-config [1]

- Playbook automation config. [[https://github.com/advthreat/tenzin-config/pull/1037][#1037]]