#+title: FY24Q2 Report #+subtitle: logs goes 4 months back #+date: 2024-01-26 #+options: H:6 ^:nil * IROH ** lead *** [1] **** iroh [1] _between 3 and 4 months old_ - Don't fire disabled webhooks [[https://github.com/advthreat/iroh/pull/8741][#8741]] ** data *** Mario Aquino [15] **** iroh [10] - Fix disabled threat-hunt test [[https://github.com/advthreat/iroh/pull/8814][#8814]] - Update incident_time when updating incident status [[https://github.com/advthreat/iroh/pull/8801][#8801]] - incident enrichment activity diagram [[https://github.com/advthreat/iroh/pull/8712][#8712]] - Separate Risk score & incident enrichment [[https://github.com/advthreat/iroh/pull/8751][#8751]] _between 3 and 4 months old_ - Improve safe-filtering [[https://github.com/advthreat/iroh/pull/8731][#8731]] - iroh-async: Flatten Datadog context [[https://github.com/advthreat/iroh/pull/8706][#8706]] - iroh-async logging & tracing context [[https://github.com/advthreat/iroh/pull/8705][#8705]] - Socket timeout milliseconds (not seconds) [[https://github.com/advthreat/iroh/pull/8690][#8690]] - Risk Score socket-timeout [[https://github.com/advthreat/iroh/pull/8687][#8687]] - Threat Hunt Module Exclusion [[https://github.com/advthreat/iroh/pull/8646][#8646]] **** tenzin-config [5] - Increase conn-manager thread count after PROD performance monitoring [[https://github.com/advthreat/tenzin-config/pull/1042][#1042]] - Increase thread pool size for EU private intel conn mgr [[https://github.com/advthreat/tenzin-config/pull/1039][#1039]] - Increase connection mgr thread pool for NAM/EU/TEST [[https://github.com/advthreat/tenzin-config/pull/1030][#1030]] _between 3 and 4 months old_ - Increase thread pool size for engine connection manager [[https://github.com/advthreat/tenzin-config/pull/1012][#1012]] - Config for skipping Private Intel during investigation threat hunt [[https://github.com/advthreat/tenzin-config/pull/1009][#1009]] *** [3] **** ctia [1] _between 3 and 4 months old_ - fix wait_for for delete search [[https://github.com/threatgrid/ctia/pull/1399][#1399]] **** iroh [2] _between 3 and 4 months old_ - bundle import activity diagrams [[https://github.com/advthreat/iroh/pull/8708][#8708]] - scoring at bundle import [[https://github.com/advthreat/iroh/pull/8694][#8694]] *** Ambrose Bonnaire-Sergeant [16] **** ctia [10] - Fix 2XX response swagger/coercion, ban ~:return~ [[https://github.com/threatgrid/ctia/pull/1407][#1407]] - Remove asset properties/mapping merging during bundle patch [[https://github.com/threatgrid/ctia/pull/1408][#1408]] _between 3 and 4 months old_ - Fix :body descriptions [[https://github.com/threatgrid/ctia/pull/1409][#1409]] - Fix POST /bulk schema checking [[https://github.com/threatgrid/ctia/pull/1406][#1406]] - Use prn instead of pprint for logs [[https://github.com/threatgrid/ctia/pull/1401][#1401]] - Eval routes and options given to ~context~ at initialization time [[https://github.com/threatgrid/ctia/pull/1394][#1394]] - Use ~st/merge~ to merge schemas instead of ~into~ [[https://github.com/threatgrid/ctia/pull/1398][#1398]] - Never match existing asset-* entities when patch-existing=false [[https://github.com/threatgrid/ctia/pull/1395][#1395]] - Re-enable incident tests [[https://github.com/threatgrid/ctia/pull/1393][#1393]] - Add external_ids to investigation select fields [[https://github.com/threatgrid/ctia/pull/1392][#1392]] **** iroh [4] - Generate valid DI auth tokens for incident subscriptions [[https://github.com/advthreat/iroh/pull/8804][#8804]] - Fix ~(reset)~ [[https://github.com/advthreat/iroh/pull/8799][#8799]] - Subscribe to incident asset rescoring via DI [[https://github.com/advthreat/iroh/pull/8699][#8699]] _between 3 and 4 months old_ - Only pass default patch-existing query params in bundle/import proxy if patch-existing=true [[https://github.com/advthreat/iroh/pull/8725][#8725]] **** tenzin-config [2] - Add config for DI client in iroh-engine [[https://github.com/advthreat/tenzin-config/pull/1036][#1036]] - Add device insights url to iroh-engine config [[https://github.com/advthreat/tenzin-config/pull/1014][#1014]] ** integrations *** [4] **** iroh [4] _between 3 and 4 months old_ - JMX metrics for clj-http connection manager [[https://github.com/advthreat/iroh/pull/8765][#8765]] - Always decompress the body when status is not 2xx [[https://github.com/advthreat/iroh/pull/8527][#8527]] - Restore default expiration (24h) for the local session token [[https://github.com/advthreat/iroh/pull/8747][#8747]] - Workflow event schema changes [[https://github.com/advthreat/iroh/pull/8656][#8656]] *** Kirill Chernyshov [10] **** iroh [7] - Refactor data streams service [[https://github.com/advthreat/iroh/pull/8793][#8793]] - DRY'ed out client-creds-token namespace [[https://github.com/advthreat/iroh/pull/8783][#8783]] - Kafka connect monitoring [[https://github.com/advthreat/iroh/pull/8278][#8278]] _between 3 and 4 months old_ - Handle undelivered records [[https://github.com/advthreat/iroh/pull/8634][#8634]] - Events data retention enforcement job [[https://github.com/advthreat/iroh/pull/8722][#8722]] - Iroh events data retention implementation [[https://github.com/advthreat/iroh/pull/8666][#8666]] - Iroh events postgres data retention [[https://github.com/advthreat/iroh/pull/8693][#8693]] **** tenzin-config [3] - Add ES sink connector v2 to test full migration [[https://github.com/advthreat/tenzin-config/pull/1035][#1035]] - Fix broken data stream on TEST [[https://github.com/advthreat/tenzin-config/pull/1034][#1034]] - Config for DataStreams service [[https://github.com/advthreat/tenzin-config/pull/1033][#1033]] *** Shafiq [7] **** iroh [4] - Fix schema of proxy health check [[https://github.com/advthreat/iroh/pull/8827][#8827]] - Add string matching for health check [[https://github.com/advthreat/iroh/pull/8815][#8815]] - Fallback to iroh-events store when kafka send fails [[https://github.com/advthreat/iroh/pull/8786][#8786]] _between 3 and 4 months old_ - Fix mapping for incident events [[https://github.com/advthreat/iroh/pull/8703][#8703]] **** tenzin-config [3] _between 3 and 4 months old_ - Swtich to new sink-connector in INT [[https://github.com/advthreat/tenzin-config/pull/1024][#1024]] - Add updated sink connector for all envs [[https://github.com/advthreat/tenzin-config/pull/1021][#1021]] - Add sink connector for new iroh-event datastream [[https://github.com/advthreat/tenzin-config/pull/1019][#1019]] ** auth *** bartuka [26] **** iroh [22] - [IROH Auth] Support FMC in the ~jwks~ service [[https://github.com/advthreat/iroh/pull/8830][#8830]] - [IROH Auth] Fix DI onboarding in Universal Provisioning Flow [[https://github.com/advthreat/iroh/pull/8813][#8813]] - Revert "[IROH Auth] support for FMC token in JWKS Service" [[https://github.com/advthreat/iroh/pull/8816][#8816]] - [IROH Auth] support for FMC token in JWKS Service [[https://github.com/advthreat/iroh/pull/8808][#8808]] - [IROH Auth] Check QA ~callback_url~ to complete provisioning tests [[https://github.com/advthreat/iroh/pull/8763][#8763]] _between 3 and 4 months old_ - [IROH Auth] better swagger descriptions for Universal Provisioning [[https://github.com/advthreat/iroh/pull/8752][#8752]] - [IROH Auth] remove empty strings from ~client-id~ got from Vault [[https://github.com/advthreat/iroh/pull/8760][#8760]] - [IROH Auth] add missing scope to get OKTA JWT [[https://github.com/advthreat/iroh/pull/8759][#8759]] - [IROH Auth] add logs to investigate get okta jwt [[https://github.com/advthreat/iroh/pull/8758][#8758]] - [IROH Auth] bugfix - ~client/post~ should use ~:form-params~ instead of ~:body~ [[https://github.com/advthreat/iroh/pull/8753][#8753]] - [IROH Auth] bugfixes - arity exception, change ~product-response~ datatype, change ~pmap~ to ~map~ [[https://github.com/advthreat/iroh/pull/8738][#8738]] - [IROH Auth] bugfix - fix urls in ~POST /tenants~ returned value and payload field names [[https://github.com/advthreat/iroh/pull/8733][#8733]] - [IROH Auth] Cache OKTA JWT used for provisioning callback [[https://github.com/advthreat/iroh/pull/8727][#8727]] - [IROH Auth] Improvements on logs and error handling to UniversalProvisioning and JWKSService [[https://github.com/advthreat/iroh/pull/8707][#8707]] - [IROH Auth] Duplicate ~universal-provisioning~ web routes to accept IROH JWTs [[https://github.com/advthreat/iroh/pull/8675][#8675]] - [IROH Auth] Expose ~callbacks packages~ store to check Universal Provisioning status [[https://github.com/advthreat/iroh/pull/8702][#8702]] - [IROH Auth] Improve 202 Accepted response for ~/universal-provisioning/create-tenants~ [[https://github.com/advthreat/iroh/pull/8701][#8701]] - [IROH Auth] Fix name convention to callbacks route in Universal Provisioning flow [[https://github.com/advthreat/iroh/pull/8691][#8691]] - [IROH Auth] Gen OKTA JWT to callback request in Universal Provisioning flow [[https://github.com/advthreat/iroh/pull/8673][#8673]] - bugfix - missing ~UniversalProvisioningCallbackService~ to deploy IROH nodes [[https://github.com/advthreat/iroh/pull/8680][#8680]] - [IROH Auth] Check pending provisions from time-to-time. Endpoint that will be called by OPS tick [[https://github.com/advthreat/iroh/pull/8674][#8674]] - [IROH Auth] Add callback handler to receive provisioning status from downstream apps [[https://github.com/advthreat/iroh/pull/8633][#8633]] **** tenzin-config [4] - FMC base-urls to configure JWKS [[https://github.com/advthreat/tenzin-config/pull/1040][#1040]] _between 3 and 4 months old_ - [IROH Auth] bugfix - add config to okta jwks [[https://github.com/advthreat/tenzin-config/pull/1017][#1017]] - [IROH Auth] add OKTA config for Universal Provisioning flow #1010 [[https://github.com/advthreat/tenzin-config/pull/1013][#1013]] - add universal_provisioning_callbacks store [[https://github.com/advthreat/tenzin-config/pull/1011][#1011]] *** Yann Esposito [27] **** iroh [17] - Fix PIAM Universal Provisioning routes [[https://github.com/advthreat/iroh/pull/8828][#8828]] - Should fix open impersonate flaky test [[https://github.com/advthreat/iroh/pull/8809][#8809]] - Keep track of impersonators [[https://github.com/advthreat/iroh/pull/8736][#8736]] - Restrict TAC routes to admins [[https://github.com/advthreat/iroh/pull/8794][#8794]] _between 3 and 4 months old_ - Remove with-tk [[https://github.com/advthreat/iroh/pull/8779][#8779]] - Code/Test Improvements [[https://github.com/advthreat/iroh/pull/8767][#8767]] - add a test for matching schema [[https://github.com/advthreat/iroh/pull/8770][#8770]] - Custom Role Design doc [[https://github.com/advthreat/iroh/pull/8497][#8497]] - Attempt to improve error message of match? [[https://github.com/advthreat/iroh/pull/8769][#8769]] - Use ~cid~ for ~trace_id~ when present [[https://github.com/advthreat/iroh/pull/8754][#8754]] - Support public client for custom routes [[https://github.com/advthreat/iroh/pull/8749][#8749]] - Add playbook scope [[https://github.com/advthreat/iroh/pull/8739][#8739]] - Fix webhook race condition risk [[https://github.com/advthreat/iroh/pull/8728][#8728]] - Call get-org only once for org-virtual user [[https://github.com/advthreat/iroh/pull/8724][#8724]] - Use a cache for entitlement summaries [[https://github.com/advthreat/iroh/pull/8667][#8667]] - upgrade jetty version [[https://github.com/advthreat/iroh/pull/8714][#8714]] - Remove a forgotten pretty printer [[https://github.com/advthreat/iroh/pull/8713][#8713]] **** iroh-scripts [6] - save improvements _between 3 and 4 months old_ - create dashboard clients - Support client aliases in get-client - Update client with client-aliases - many new scripts - added a 1-time script **** tenzin-config [2] _between 3 and 4 months old_ - Check vault templating error [[https://github.com/advthreat/tenzin-config/pull/1023][#1023]] - Add Universal Provisioning Services [[https://github.com/advthreat/tenzin-config/pull/1015][#1015]] **** xdr-provisioning [2] _between 3 and 4 months old_ - Manage entitlements add-ons - add a re-provisioning script that rerun onboardings *** Olivier Barbeau [25] **** iroh [15] - Check the list of services for a node type [[https://github.com/advthreat/iroh/pull/8800][#8800]] - Fix merge error on PR 8784 [[https://github.com/advthreat/iroh/pull/8797][#8797]] - [IROH configuration]: Move role-web-service config to default tk files [[https://github.com/advthreat/iroh/pull/8782][#8782]] - [IROH configuration]: Universal Provisioning Services config refactor [[https://github.com/advthreat/iroh/pull/8784][#8784]] - [IROH configuration]: explicit name for generated conf and meta [[https://github.com/advthreat/iroh/pull/8785][#8785]] - Clean bootstrap.cfg; remove tmp file [[https://github.com/advthreat/iroh/pull/8781][#8781]] _between 3 and 4 months old_ - Add few additional tests to iroh services [[https://github.com/advthreat/iroh/pull/8762][#8762]] - rewrite tests [[https://github.com/advthreat/iroh/pull/8773][#8773]] - more info for debugging [[https://github.com/advthreat/iroh/pull/8717][#8717]] - [IROH configuration]: general documentation [[https://github.com/advthreat/iroh/pull/8764][#8764]] - update developer doc for api-gateway [[https://github.com/advthreat/iroh/pull/8723][#8723]] - Some test clean-up [[https://github.com/advthreat/iroh/pull/8716][#8716]] - High volume of SQL queries for a single observe/deliberate call [[https://github.com/advthreat/iroh/pull/8682][#8682]] - Remove the state of module instances in ~obfuscate-module-instance~ [[https://github.com/advthreat/iroh/pull/8670][#8670]] - E8388: update proxy-endpoints-metadata endpoint and metadata [[https://github.com/advthreat/iroh/pull/8663][#8663]] **** tenzin-config [10] - Deep merge for vectors and sets with duplicates check [[https://github.com/advthreat/tenzin-config/pull/1032][#1032]] - Reduce configuration duplicates - config.edn part [[https://github.com/advthreat/tenzin-config/pull/1031][#1031]] - Reduce configuration duplicates - bootstrap.cfg part [[https://github.com/advthreat/tenzin-config/pull/1028][#1028]] - Move role-web-service config to IROH [[https://github.com/advthreat/tenzin-config/pull/1026][#1026]] - Move Universal Provisioning Services config to IROH [[https://github.com/advthreat/tenzin-config/pull/1027][#1027]] - Clean bootstrap cfg [[https://github.com/advthreat/tenzin-config/pull/1025][#1025]] _between 3 and 4 months old_ - Remove the ~:registration~ flag in all environments [[https://github.com/advthreat/tenzin-config/pull/963][#963]] - Remove the ~:merge-users-by-email~ flag in all environments [[https://github.com/advthreat/tenzin-config/pull/962][#962]] - Remove the ~:account-activation-optim~ flag in all environments as it is now activated everywhere. [[https://github.com/advthreat/tenzin-config/pull/961][#961]] - Remove the ~xdr-roles~ flag in all environments [[https://github.com/advthreat/tenzin-config/pull/964][#964]] *** (Yogsototh) [8] **** iroh-scripts [6] - save improvements _between 3 and 4 months old_ - create dashboard clients - Support client aliases in get-client - Update client with client-aliases - many new scripts - added a 1-time script **** xdr-provisioning [2] _between 3 and 4 months old_ - Manage entitlements add-ons - add a re-provisioning script that rerun onboardings ** iroh-ops *** [0] *** [0] * Other ** Other *** II [4] **** iroh [4] - Merges module type props on create and update health check [[https://github.com/advthreat/iroh/pull/8845][#8845]] - Implements v2 threat hunting [[https://github.com/advthreat/iroh/pull/8833][#8833]] - This should fix issue with parent settings not used on create-patch [[https://github.com/advthreat/iroh/pull/8822][#8822]] _between 3 and 4 months old_ - Adds insights scope to allowed Automation scopes in gen-ao-jwt [[https://github.com/advthreat/iroh/pull/8678][#8678]] *** [4] **** iroh [4] - Merges module type props on create and update health check [[https://github.com/advthreat/iroh/pull/8845][#8845]] - Implements v2 threat hunting [[https://github.com/advthreat/iroh/pull/8833][#8833]] - This should fix issue with parent settings not used on create-patch [[https://github.com/advthreat/iroh/pull/8822][#8822]] _between 3 and 4 months old_ - Adds insights scope to allowed Automation scopes in gen-ao-jwt [[https://github.com/advthreat/iroh/pull/8678][#8678]] *** Scott McLeod [1] **** iroh [1] - tk store: Add delete-search method #8213 [[https://github.com/advthreat/iroh/pull/8692][#8692]] *** Brooke Swanson [3] **** ctia [1] _between 3 and 4 months old_ - Add brookeswanson to codeowners. [[https://github.com/threatgrid/ctia/pull/1396][#1396]] **** iroh [1] _between 3 and 4 months old_ - Add no-doc true and prevent explosion due to mismatched types. [[https://github.com/advthreat/iroh/pull/8548][#8548]] **** tenzin-config [1] - Playbook automation config. [[https://github.com/advthreat/tenzin-config/pull/1037][#1037]]