.cj} UdZddlZddlZddlZddlZddlZddlZddlmZm Z ddl m Z m Z m Z mZmZmZmZddlmZejeZdedefdZehd Zdede efd Zdedefd Zed d hZeejdejdejdejdejdejdejdejdejdh Z ej!dej!dfZ"edhZ#dZ$ej!dZ%da&da'ee(d<defdZ)dCdZ*d ej+ej,zdefd!Z-dedefd"Z.d#ed$edefd%Z/dedefd&Z0dedefd'Z1Gd(d)e2Z3d*ed+e4d,e5e6ee4fefdefd-Z7d*ed+e4d$ede8efd.Z9Gd/d0Z:Gd1d2Z;d3ede5e6ee4feffd4Zd8ed9eddfd:Z?d8ed9eddfd;Z@dZBd5edefd?ZCd5edefd@ZDdAede efdBZEdS)DuURL safety checks — blocks requests to private/internal network addresses. Prevents SSRF (Server-Side Request Forgery) where a malicious prompt or skill could trick the agent into fetching internal resources like cloud metadata endpoints (169.254.169.254), localhost services, or private network hosts. The check can be globally disabled via ``security.allow_private_urls: true`` in config.yaml for environments where DNS resolves external domains to private/benchmark-range IPs (OpenWrt routers, corporate proxies, VPNs that use 198.18.0.0/15 or 100.64.0.0/10). Even when disabled, cloud metadata hostnames (metadata.google.internal, 169.254.169.254) are **always** blocked — those are never legitimate agent targets. Limitations: - DNS rebinding (TOCTOU): an attacker-controlled DNS server with TTL=0 can return a public IP for the check, then a private IP for the actual connection. Hermes-owned direct httpx request paths should use ``create_ssrf_safe_client()`` / ``create_ssrf_safe_async_client()`` so the same policy is applied immediately before TCP connect and the client connects to the validated IP while preserving Host/SNI semantics. - Redirect-based bypass is mitigated by httpx event hooks that re-validate each redirect target in vision_tools, gateway platform adapters, and media cache helpers. Web tools use third-party SDKs (Firecrawl/Tavily) where redirect handling is on their servers. N)AnyOptional) parse_qslquoteunquoteurljoinurlparseurlsplit urlunsplit)is_truthy_valueurlreturnct|ts|S|}|s|Stjdd|} t |}n#t $r|cYSwxYw|jdvr|S|j }|j }|rY | d d}n#t$r|}YnwxYw||kr|||d}t|jd}t|jd }t|jd }t'|j||||fS) uReturn an ASCII-safe HTTP URL for Hermes-owned URL tools. Browsers and HTTP clients expect URIs, but users and models often provide IRIs such as ``https://wttr.in/Köln``. Preserve URL syntax and existing percent escapes while encoding non-ASCII host/path/query/fragment text. This is intentionally for URL tool inputs only; arbitrary shell commands must not be rewritten. z ^([A-Za-z][A-Za-z0-9+.-]*://)\s+z\1>httphttpsidnaasciiz/%:@!$&'()*+,;=)safez/%:@!$&'()*+,;=?) isinstancestrstripresubr ValueErrorschemelowernetlochostnameencodedecode UnicodeErrorreplacerpathqueryfragmentr ) r rawparsedrr ascii_hostr$r%r&s 2/home/ice/.hermes/hermes-agent/tools/url_safety.pynormalize_url_for_requestr+*s| c3   ))++C   &4eS A AC#  }$555 ]FH= "!0077@@JJ " " "!JJJ "  ! !^^Hj!<>>H v}fdE8D E EEs$A A&%A&(C CC>x-amz-signaturex-amz-security-tokenjwttokenapikeypasswdsecretapi_keypassword signature auth_token credential session_id credentials access_token authorization client_secretawsaccesskeyidx_amz_signaturex_amz_security_tokenct|trd|vrdS t|}n#t$rYdSwxYw|jdvs|jsdSt|jdD]3\}}|r,t|tvr|cS4dS)alReturn the first sensitive query parameter name in ``url``, if any. Used before handing URLs to third-party fetch/browser backends. Prefix-based token redaction catches known credential shapes; this catches opaque magic links, OAuth codes, signed URL signatures, and custom ``?token=...`` values that do not have a recognizable vendor prefix. ?N>rrT)keep_blank_values) rrr rrrrr%rr_SENSITIVE_QUERY_PARAM_NAMES)r r(keyvalues r*sensitive_query_param_namerFys c3  3c>>t#))++&& tt }$555V\5t EEE U  WS\\''))-IIIJJJ 4s!? A  A c$t|duS)zHReturn True when ``url`` carries likely credential-bearing query params.N)rFr s r*has_sensitive_query_paramsrIs %c * *$ 66zmetadata.google.internalz metadata.googz169.254.169.254z 169.254.170.2z169.254.169.253z fd00:ec2::254z100.100.100.200z::ffff:169.254.169.254z::ffff:169.254.170.2z::ffff:169.254.169.253z::ffff:100.100.100.200z169.254.0.0/16z::ffff:169.254.0.0/112zmultimedia.nt.qq.com.cnz 100.64.0.0/10F_cached_allow_privatecztrtSdadatjdd}|dvr datS|dvrtS ddlm}|}|d i}t|tr-t|d d r datS|d i}t|tr-t|d d r datSn#t$rYnwxYwtS) acReturn True when the user has opted out of private-IP blocking. Checks (in priority order): 1. ``HERMES_ALLOW_PRIVATE_URLS`` env var (``true``/``1``/``yes``) 2. ``security.allow_private_urls`` in config.yaml 3. ``browser.allow_private_urls`` in config.yaml (legacy / backward compat) Result is cached for the process lifetime. TFHERMES_ALLOW_PRIVATE_URLS>1yestrue>0nofalser)read_raw_configsecurityallow_private_urls)defaultbrowser) _allow_private_resolvedrLosgetenvrrhermes_cli.configrVgetrdictr Exception)env_valrVcfgsecrZs r*_global_allow_private_urlsresw%$$"!i3R88>>@@FFHHG&&& $$$&&&$$ 555555oggj"%% c4  )_ GG( ) )5& & &  )%) !( ('')R(( gt $ $ ) KK, - -u* * *  )%) !( (       ! s%A'D& AD&& D32D3cdadadS)u+Reset the cached toggle — only for tests.FN)r[rLrJr*_reset_allow_private_cacherhs$!rJipc.t|tjrA|j:|j}|jp+|jp$|jp|jp|jp|j p|tvS|js|js|js|jrdS|js|j rdS|tvrdSdS)zz(is_always_blocked_url..Ns800!c 000000rJzDBlocked request to cloud metadata address (always-blocked floor): %sr%u>Unparseable IP address %r for hostname %s — skipping addressc3 K|]}|vV dSrzrg)r|r}resolveds r*r~z(is_always_blocked_url..ks866$'C666666rJzJBlocked request to cloud metadata address (always-blocked floor): %s -> %sz&is_always_blocked_url error for %s: %s)r rrrrstrip_BLOCKED_HOSTNAMESloggerwarningrk ip_addressr_ALWAYS_BLOCKED_IPSany_ALWAYS_BLOCKED_NETWORKSsocket getaddrinfo AF_UNSPEC SOCK_STREAMgaierrorsplitradebug) r r(r addr_info_family_sockaddrip_strexcrirs @@r*is_always_blocked_urlrs:C#O)r002288::AA#FF 5 ) ) ) NNQ   4 %h//BB   BBB  >(((C0000%=000--(1 t5 *$ 0&2DII   55 +4   &GQ1ha[Ff}}c**1- $/77   _aijkalnvwww ...#6666+C66633.7  tt/u   =sCHHHuuuuu sAG6$G6>BG6 B"G6!B""A G60+DG6D/+G6.D//4G6$E98G69,F(%G6'F((A G63G66 H'H""H'rrc |dko|tvS)zGReturn True when a trusted HTTPS hostname may bypass IP-class blocking.r)_TRUSTED_PRIVATE_IP_HOSTS)rrs r*_allows_private_ip_resolutionrs W  F-F!FFrJc^ t|}|jpdd}|jpd}|dvrt d|pddS|sdS|tvrt d|dSt}t||} tj |dtj tj}n1#tj$rt d |YdSwxYw|D]\}}}}} | d } d | vr| d d } t#j|  n4#t&$r't d | d |YdSwxYw t(vs t+ fd t,Drt d|| dS|s0|s.t/ rt d|| dS|rt d|n|rt d|dS#t2$r'} t d|| Yd} ~ dSd} ~ wwxYw)uReturn True if the URL target is not a private/internal address. Resolves the hostname to an IP and checks against private ranges. Fails closed: DNS errors and unexpected exceptions block the request. When ``security.allow_private_urls`` is enabled (or the env var ``HERMES_ALLOW_PRIVATE_URLS=true``), private-IP blocking is skipped. Cloud metadata endpoints (169.254.169.254, metadata.google.internal) remain blocked regardless — they are never legitimate agent targets. rOrx>rru.Blocked request — unsupported URL scheme: %szFz(Blocked request to internal hostname: %sNu1Blocked request — DNS resolution failed for: %srru=Blocked request — unparseable IP address %r for hostname %sc3 K|]}|vV dSrzrgr{s r*r~zis_safe_url..s'/^/^cc /^/^/^/^/^/^rJz3Blocked request to cloud metadata address: %s -> %sz5Blocked request to private/internal address: %s -> %szKAllowing private/internal resolution (security.allow_private_urls=true): %szAAllowing trusted hostname despite private/internal resolution: %sTu5Blocked request — URL safety check error for %s: %s)r rrrrrrrrrerrrrrrrrkrrrrrrvrra) r r(rrallow_all_privateallow_private_iprfamilyrrrrris @r* is_safe_urlrs#H#O)r002288::AA#FF-%2,,..4466 * * * NNKVM`W` a a a5 5 ) ) ) NNEx P P P578886JJ *8T6;KVM_``II    NNNPX Y Y Y55   *3   %FAq!Xa[Ff}}c**1- )&11   ^`hij`kmuvvvuuu  (((C/^/^/^/^E]/^/^/^,^,^(Ifuu$ -= .QSBTBT Kfuu   LL]      LLS    t  NPSUXYYYuuuuu sBI;"I;&$I; I;++DI;*EI;E4I;:FI;,G;I;?GA I; 0I;=.s'+Z+Z#B#I+Z+Z+Z+Z+Z+ZrJz:Blocked request to cloud metadata address during connect: z -> zP>P 'exee]cee    #h--2G"G"G OOF # # # HHV    b#$`V^$`$`aaa Os0+B44CCC!D66 E!EE!c eZdZdefdZ ddedededzdedzded ef d Z dd ededzded efd Z d ed dfdZ dS)_SSRFGuardedAsyncNetworkBackendschemes_by_origin_varc>ddlm}||_||_dS)Nr) AutoBackend)httpcore._backends.autor_backend_schemes_by_origin_var)selfrrs r*__init__z(_SSRFGuardedAsyncNetworkBackend.__init__'/777777#  &;###rJNrrtimeout local_addresssocket_optionsrcpKddl}|ji}t|||}t jt |||d{V} d} | D]I} |j| ||||d{VcS#|j |j f$r } | } Yd} ~ Bd} ~ wwxYw| | td|Nr)rrrz2Blocked request - DNS returned no usable IPs for: ) httpcorerr_rrrrr connect_tcp ConnectErrorConnectTimeoutr rrrrrrrrripslast_excrirs r*rz+_SSRFGuardedAsyncNetworkBackend.connect_tcp-s3  7;;B??%dD2CDD%&@$fUUUUUUUU%)  B !]66#"/#1 7)8+BC      N#$_Y]$_$_```s$BBBBr$c$KtdNz5Blocked Unix socket connection in SSRF-safe transportrrr$rrs r*connect_unix_socketz3_SSRFGuardedAsyncNetworkBackend.connect_unix_socketLs $$[\\\rJsecondscJK|j|d{VdSrzrsleeprrs r*rz%_SSRFGuardedAsyncNetworkBackend.sleepTs4m!!'***********rJNNNNN rrrrrrintfloatrrrrgrJr*rr&sddlm}||_||_dS)Nr) SyncBackend)httpcore._backends.syncrrr)rrrs r*rz#_SSRFGuardedNetworkBackend.__init__YrrJNrrrrrrc>ddl}|ji}t|||}t |||} d} | D]C} |j| ||||cS#|j|jf$r } | } Yd} ~ rr)r rrr)rrrrs r*_origin_scheme_contextrsK ; D ; D [ F 4<61B#B#B 4L& !!rJkwargsc xddl}ddl}|dGfdd|j}|di|S)zLReturn an httpx async transport that pins direct TCP connects to vetted IPs.rN hermes_ssrf_async_origin_schemesc>eZdZdeffd Zdedeffd ZxZS)2ssrf_safe_async_http_transport.._Transporttransport_kwargscltjdi|t|j_dSNrg)superrr_pool_network_backendrr __class__rs r*rz;ssrf_safe_async_http_transport.._Transport.__init__s> EGG  0 0/ 0 0 0*I%++DJ ' ' 'rJrrcKt|} t|d{V |S#|wxYwrz)rrrhandle_async_requestresetrrr/rrs r*rzGssrf_safe_async_http_transport.._Transport.handle_async_requests)--.DW.M.MNNE 3"WW99'BBBBBBBBB%++E2222%++E2222s &A$$A;)rrrrrr __classcell__rrs@r* _Transportrs{ s         3c 3c 3 3 3 3 3 3 3 3 3 3 3rJrrg) contextvarshttpx ContextVarAsyncHTTPTransportrrrrrs @r*ssrf_safe_async_http_transportrs|LLL'223UVV 3 3 3 3 3 3 3U- 3 3 3 :    rJc xddl}ddl}|dGfdd|j}|di|S)zKReturn an httpx sync transport that pins direct TCP connects to vetted IPs.rNhermes_ssrf_origin_schemesc>eZdZdeffd Zdedeffd ZxZS),ssrf_safe_http_transport.._Transportrcltjdi|t|j_dSr)rrrrrrs r*rz5ssrf_safe_http_transport.._Transport.__init__s> EGG  0 0/ 0 0 0*D%++DJ ' ' 'rJrrct|} t||S#|wxYwrz)rrrhandle_requestrrs r*r z;ssrf_safe_http_transport.._Transport.handle_requestsk)--.DW.M.MNNE 3ww--g66%++E2222%++E2222s AA2)rrrrrr rrs@r*rrs{ s         3# 3# 3 3 3 3 3 3 3 3 3 3 3rJrrg)rrr HTTPTransportrs @r*ssrf_safe_http_transportr s|LLL'223OPP 3 3 3 3 3 3 3U( 3 3 3 :    rJ transportrc|t|dini}||ddrdS|d}|t|dstdt |_t|ddtddt d t ffd }||_d |_dS) N__dict___hermes_ssrf_guardedFrrz:Unsupported async httpx transport cannot be made SSRF-saferrrcKt|} |d{V |S#|wxYwrzrrr)rr/rrs r*guarded_handle_async_requestzL_install_ssrf_guard_on_async_transport..guarded_handle_async_requests}%))*@*I*IJJ /--g66666666 6 ! ' ' . . . . ! ' ' . . . .s AA%T) getattrr_hasattrrrrrrr)rrstatepoolrrs ` @r*&_install_ssrf_guard_on_async_transportrs2;2GGIz2 . . .REEII&.guarded_handle_requestsc%))*@*I*IJJ /!>'** ! ' ' . . . . ! ' ' . . . .s AAT) rr_rrrrrr r)rrrrrr s ` @r* _install_ssrf_guard_on_transportrs2;2GGIz2 . . .REEII&r?s-6  [[[[[[[[[[[[[[[[[[!!!!!!  8 $ $-F3-F3-F-F-F-Fn )y***  0CHSM,7C7D7777Y  iI*++I))I*++I))I*++I122I/00I122I122 !  I)**I122&I' &%o66  #t###0!D0!0!0!0!f""""y,y/DD,`s`t````FGCGGGGGG SSSTSSSSl555555RRRRRJRRRWsW#W$uSRUXX[G[B\WadWWWW8S88S8T#Y8888v/+/+/+/+/+/+/+/+d/%/%/%/%/%/%/%/%d"C"DsCx#1E,F"""" S S    0 s s    0*c*RU*Z^****2**C*TX****2#$ C C     ccCHSMrJ