[jT`UdZddlmZddlZddlZddlZddlZddlZddlm Z ddl m Z m Z m Z mZmZejeZdZded<d Zd Zded <d ZGd dejZd>dZdddd?dZdddd@dZddddd Zdd!d"d#dAd)Zd*d+ddd,dBd6Zd7d+d+d+d+d8dCd;Z Gd<d=eZ!dS)Du Video Generation Provider ABC ============================= Defines the pluggable-backend interface for video generation. Providers register instances via ``PluginContext.register_video_gen_provider()``; the active one (selected via ``video_gen.provider`` in ``config.yaml``) services every ``video_generate`` tool call. Providers live in ``/plugins/video_gen//`` (built-in, auto-loaded as ``kind: backend``) or ``~/.hermes/plugins/video_gen//`` (user, opt-in via ``plugins.enabled``). Mirrors the ``image_gen`` provider design (``agent/image_gen_provider.py``) so the two surfaces stay learnable together. Unified surface --------------- One tool — ``video_generate`` — covers **text-to-video** and **image-to-video**. The router is the presence of ``image_url``: if it's set, the provider routes to its image-to-video endpoint; if it's omitted, the provider routes to text-to-video. Users pick one **model family** (e.g. Pixverse v6, Veo 3.1, Kling O3 Standard); the provider handles which underlying FAL/xAI endpoint to hit. Video edit and video extend are intentionally NOT exposed in this surface — the inconsistency across backends is too large for one unified tool. If those use cases warrant attention later they can ship as separate tools. Response shape -------------- All providers return a dict built by :func:`success_response` / :func:`error_response`. Keys: success bool video str | None URL or absolute file path model str provider-specific model identifier prompt str echoed prompt modality str "text" | "image" (which mode was used) aspect_ratio str provider-native (e.g. "16:9") or "" duration int seconds (0 if not applicable) provider str provider name (for diagnostics) error str only when success=False error_type str only when success=False ) annotationsN)Path)AnyDictListOptionalTuple)16:9z9:16z1:1z4:3z3:4z3:2z2:3zTuple[str, ...]COMMON_ASPECT_RATIOSr )480p540p720p1080pCOMMON_RESOLUTIONSrc eZdZdZeejd!dZed!dZd"dZ d#d Z d$d Z d%d Z d$dZ ejddddeedddd d&d ZdS)'VideoGenProvideruAbstract base class for a video generation backend. Subclasses must implement :meth:`generate`. Everything else has sane defaults — override only what your provider needs. returnstrcdS)zStable short identifier used in ``video_gen.provider`` config. Lowercase, no spaces. Examples: ``xai``, ``fal``, ``google``. Nselfs :/home/ice/.hermes/hermes-agent/agent/video_gen_provider.pynamezVideoGenProvider.nameRc4|jS)zMHuman-readable label shown in ``hermes tools``. Defaults to ``name.title()``.)rtitlers r display_namezVideoGenProvider.display_nameZsy   rboolcdS)zReturn True when this provider can service calls. Typically checks for a required API key and optional-dependency import. Default: True. Trrs r is_availablezVideoGenProvider.is_available_s trList[Dict[str, Any]]cgS)aReturn catalog entries for ``hermes tools`` model picker. Each entry represents a **model family** that supports text-to-video and/or image-to-video routing internally:: { "id": "veo-3.1", # required "display": "Veo 3.1", # optional; defaults to id "speed": "~60s", # optional "strengths": "...", # optional "price": "$0.20/s", # optional "modalities": ["text", "image"], # optional, advisory } Default: empty list (provider has no user-selectable models). rrs r list_modelszVideoGenProvider.list_modelsgs " rDict[str, Any]c|jddgdS)z9Return provider metadata for the ``hermes tools`` picker.)rbadgetagenv_vars)rrs rget_setup_schemaz!VideoGenProvider.get_setup_schemazs"%    r Optional[str]ch|}|r|ddSdS)z7Return the default model id, or None if not applicable.ridN)r%get)rmodelss r default_modelzVideoGenProvider.default_models6!!##  '!9==&& &trc bdgttttddddddS)aReturn what this provider supports. Returned dict (all keys optional):: { "modalities": ["text", "image"], # which inputs the backend accepts "aspect_ratios": ["16:9", "9:16", ...], "resolutions": ["720p", "1080p"], "max_duration": 15, # seconds "min_duration": 1, "supports_audio": True, "supports_negative_prompt": True, "max_reference_images": 7, } Used by the tool layer for soft validation and by ``hermes tools`` for the picker. Default: text-only. text Fr) modalities aspect_ratios resolutions max_duration min_durationsupports_audiosupports_negative_promptmax_reference_images)listr rrs r capabilitieszVideoGenProvider.capabilitiess?("(!"677 233#(-$%   rN model image_urlreference_image_urlsduration aspect_ratio resolutionnegative_promptaudioseedpromptrBrCrDOptional[List[str]]rE Optional[int]rFrGrHrIOptional[bool]rJkwargsrc  dS)uGenerate a video from a prompt (text-to-video) or animate an image (image-to-video). Routing: if ``image_url`` is provided, the provider should route to its image-to-video endpoint; otherwise text-to-video. The plugin is responsible for picking the right underlying endpoint within the user's chosen model family. Implementations should return the dict from :func:`success_response` or :func:`error_response`. ``kwargs`` may contain forward-compat parameters future versions of the schema will expose — implementations MUST ignore unknown keys (no TypeError). Nr) rrKrBrCrDrErFrGrHrIrJrOs rgeneratezVideoGenProvider.generaterrrrrr )rr#)rr&)rr-rKrrBr-rCr-rDrLrErMrFrrGrrHr-rIrNrJrMrOrrr&)__name__ __module__ __qualname____doc__propertyabcabstractmethodrrr"r%r,r2r@DEFAULT_ASPECT_RATIODEFAULT_RESOLUTIONrQrrrrrKs    X !!!X!&        <  $#'48"&0,)- $"        rrrrc`ddlm}|dz dz }|dd|S)zBReturn ``$HERMES_HOME/cache/videos/``, creating parents as needed.r)get_hermes_homecachevideosT)parentsexist_ok)hermes_constantsr_mkdir)r_paths r_videos_cache_dirrgsF000000 ?  w & 1DJJtdJ+++ Krvideomp4)prefix extensionb64_datarrjrkc2tj|}tjd}t jjdd}t|d|d|d|z }| ||S)zDecode base64 video data and write under ``$HERMES_HOME/cache/videos/``. Returns the absolute :class:`Path` to the saved file. Filename format: ``__.``. %Y%m%d_%H%M%SN_.) base64 b64decodedatetimenowstrftimeuuiduuid4hexrg write_bytes)rlrjrkrawtsshortrfs rsave_b64_videor~s  8 $ $C     ) )/ : :B JLL RaR E   F!E!ER!E!E%!E!E)!E!E EDS Krr{bytesc tjd}tjjdd}t |d|d|d|z }|||S)z@Write raw video bytes (e.g. an HTTP download body) to the cache.rnNrorprq)rtrurvrwrxryrgrz)r{rjrkr|r}rfs rsave_bytes_videors     ) )/ : :B JLL RaR E   F!E!ER!E!E%!E!E)!E!E EDS Krwebmmovmkv)z video/mp4z video/webmzvideo/quicktimezvideo/x-matroskagf@i )rjtimeout max_bytesurlrfloatrintc ddl}|||d}||jdpdddd}t|}|O|d dd}d D]} |d | r| }n|d }tj d } tj jdd} t|d| d| d |z } d} | d5}|dD]}|s| t%|z } | |krS| | n#t*$rYnwxYwt-d|d|dzd|| dddn #1swxYwY| dkr9 | n#t*$rYnwxYwt-d|d| S)aDownload a video URL and write it under ``$HERMES_HOME/cache/videos/``. The video twin of :func:`agent.image_gen_provider.save_url_image`: several backends (DeepInfra, FAL) return an *ephemeral* delivery URL that expires before a downstream consumer can fetch it, so we materialise the bytes locally at tool-completion time. Streams with a size cap. Raises on any network / HTTP / oversize error so callers can fall back to returning the bare URL. rNT)rstreamz Content-Typer(;r6?)rirrrrqrirnrorpwbi) chunk_sizez Video at z exceeds izMB cap; refusing to cache.z was empty (0 bytes).)requestsr0raise_for_statusheaderssplitstriplower_URL_VIDEO_CONTENT_TYPESendswithrtrurvrwrxryrgopen iter_contentlencloseunlinkOSError ValueErrorwrite)rrjrrrresponse content_typerkurl_pathextr|r}rf bytes_writtenfhchunks rsave_url_videors"OOO||C|>>H $((88>BEEc1MMaPVVXX^^``L(,,\::I99S!$$Q'--//0  C  S++        ) )/ : :B JLL RaR E   F!E!ER!E!E%!E!E)!E!E EDM 4B**j*AA  E  SZZ 'My(( KKMMMMD dddiK.Hddd HHUOOOO    KKMMMM    D ?S???@@@ KsI>AH(GH( G)&H((G))2H((H,/H,:I IIr4r()modalityrFrEextrarBrKrrFrEproviderrOptional[Dict[str, Any]]r&c d||||||rt|nd|d}|r0|D]\} } || | |S)uBuild a uniform success response dict. ``video`` may be an HTTP URL or an absolute filesystem path. ``modality`` is ``"text"`` (text-to-video) or ``"image"`` (image-to-video) — indicates which endpoint was actually hit, useful for diagnostics. Tr)successrhrBrKrrFrEr)ritems setdefault) rhrBrKrrFrErrpayloadkvs rsuccess_responser?sz$$%-4CMMM1  G %KKMM % %DAq   q! $ $ $ $ Nrprovider_error) error_typerrBrKrFerrorrc dd||||||dS)z$Build a uniform error response dict.FN)rrhrrrBrKrFrrrrrrBrKrFs rerror_responser`s+ $   rc eZdZUdZdZded<dZded<dZded <d Zded <d(d Z d)dZ d*dZ d(dZ dddde edddd d+d'ZdS), OpenAICompatibleVideoGenProviderugGeneric text/image-to-video over the OpenAI ``client.videos`` API. DeepInfra, OpenAI/Sora, and OpenRouter all expose the same ``POST /videos`` async-job shape (``create`` → poll → ``download_content``), so the SDK call lives here once. A concrete backend only needs to declare its identity and credentials:: class FooVideoGenProvider(OpenAICompatibleVideoGenProvider): name = "foo" _env_key = "FOO_API_KEY" _default_base_url = "https://api.foo.com/v1/openai" def list_models(self): return [...] # entries with an "id" key; default_model() uses [0] ``image_url`` routes to image-to-video; its absence routes to text-to-video. Provider-specific fields (``image_url``/``negative_prompt``/``seed``) ride in ``extra_body`` so they pass through the SDK unchanged. OPENAI_API_KEYr_env_keyzhttps://api.openai.com/v1_default_base_urlg@r_poll_interval_sg @_poll_deadline_srcnddl}|j|jdS)Nrr()osenvironr0rr)rrs r_api_keyz)OpenAICompatibleVideoGenProvider._api_keys/ z~~dmR0066888rr cDt|SN)r rrs rr"z-OpenAICompatibleVideoGenProvider.is_availablesDMMOO$$$rclientr call_kwargsr&c ddl}|jjd i|}hd}||jz}t |dd|vr||krIt dt |dddt|jd t |ddd ||j |j |j }t |dd|v|S) aCCreate the video job and poll to completion with a hard deadline. Replaces ``client.videos.create_and_poll`` (unbounded 1/s loop) with a coarse interval and a wall-clock cap. Returns the terminal video object (any status); raises :class:`TimeoutError` if the deadline passes first. rN>rfailedcanceled cancelled completed succeededstatusz video job r/rz( did not reach a terminal status within zs (last status=)r) timeracreate monotonicrgetattr TimeoutErrorrsleeprretriever/)rrrrrhterminaldeadlines r_create_and_pollz1OpenAICompatibleVideoGenProvider._create_and_polls0  $ $33{33YYY>>##d&;;eXt,,H<<~~8++"Hc!:!:HH%()>%?%?HH$+E8T$B$BHHH JJt, - - -M**5844EeXt,,H<< rcddl}|j|jdd}|p|jS)Nr _BASE_URLr()rrr0rupperrr)rroverrides r _base_urlz*OpenAICompatibleVideoGenProvider._base_urlsO :>>TY__%6%6"A"A"A2FFLLNN1411rNrArKrBr-rCrDrLrErMrFrGrHrIrNrJrOc |r|stdd|jS|st|jdd|jS ddl} n'#t $rtdd|jcYSwxYw|p|} | s td |jd d |jSd |||| d D}| |d}|rt||d<|r||d<|r||d<| || } | ||}n#t$r{}td|jdt|jd|d|j| ||cYd}~t!|dd}t#|r |SSd}~wwxYwt!|dd}|dvrjt!|dd}t|rt|nd|d|j| ||t!|dd}t#|r |SSd}t!|ddpgD]C}t%|t&r|d nt!|d d}|r|}nD |r$tt+||j!}nT|j|j}tt5||j!}n#t$r}|r$td"|j||}nTt|jd#|d$|j| ||cYd}~t!|dd}t#|r |SSYd}~nd}~wwxYwt7|| ||rd%nd&||pd|j't!|dd}t#|r |SS#t!|dd}t#|r |wwxYw)(Nzprompt is requiredinvalid_request)rrrz is not setmissing_credentialsrz8openai Python package not installed (pip install openai)missing_dependencyzno z, video model available (live catalog empty?)no_modelci|] \}}||| Srr).0rrs r z=OpenAICompatibleVideoGenProvider.generate..s/    1 } q}}r)rHrFrCrJ)rBrKsecondssize extra_body)api_keybase_urlz%s video generation failedT)exc_infoz video generation failed: api_errorrrr)rrrzvideo job ended with status= job_faileddatar)rjz3%s: saving video locally failed (%s); returning URLz7 video job succeeded but no output could be retrieved: empty_responseimager4)rhrBrKrrFrEr)rrrrropenai ImportErrorr2rrOpenAIrr Exceptionloggerdebugrcallable isinstancedictr0rradownload_contentr/readrr)rrKrBrCrDrErFrGrHrIrJrOrmodel_idrrrrhexcrr job_errorritem candidate video_refr{s rrQz)OpenAICompatibleVideoGenProvider.generates V\\^^ !*7HSWS\ }} !3330    MMMM   !P/    0D..00 !SDISSS%    $3 ,&  egg    19F&K&K  3%(]]K "  -",K   3(2K %t}}AQAQRRN  --fkBB    949t TTT%!YGG#GG*!Y"!!- NFGT22E  U UHd33F777$E7D99 %,5d#i...;dZ`;d;d+!Y"!!- lFGT22E  SCvt44:  /9$/E/EeDHHUOOO7SWY^`dKeKe #CE M #N3ty$I$I$I J JII!-88BBGGIIC #$4S$K$K$K L LI    LL!VX\Xacfggg #II)!%hhcfhh#3!%&%%1 &FGT22E  -IIII $$-96)!QFGT22E  FGT22E  s"A''!B  B  E$#P$ G).AG$3G)4P$G))AP+APA:M?P O A O OPP O#P-Q rRrS)rrrr&rrrT)rUrVrWrXr__annotations__rrrrr"rrr\r]rQrrrrr{s&%H$$$$88888"!!!!#####9999 %%%%02222 $#'48"&0,)- $"MMMMMMMMrr)rr)rlrrjrrkrrr)r{rrjrrkrrr) rrrjrrrrrrr)rhrrBrrKrrrrFrrErrrrrrr&)rrrrrrrBrrKrrFrrr&)"rX __future__rrZrrrtloggingrwpathlibrtypingrrrrr getLoggerrUrr rr\rr]ABCrrgr~rrrrrrrrrr sL,,,\#"""""  33333333333333  8 $ $ )\[[[[&GGGGGy y y y y swy y y B .       & ======J&*H'6SSSSS'7SSSSSr