diff --git a/.changeset/motion-export-bvh-gltf.md b/.changeset/motion-export-bvh-gltf.md new file mode 100644 index 0000000..2e8c44b --- /dev/null +++ b/.changeset/motion-export-bvh-gltf.md @@ -0,0 +1,5 @@ +--- +"posecode-render": patch +--- + +Add motion export. `exportBVH(ir, options?)` bakes a movement's authored joint motion and root travel/turn into a standard Biovision Hierarchy (`.bvh`) file, and `exportGLTF(ir, options?)` / `buildAnimatedRig(ir, options?)` export the mannequin rig plus a baked `AnimationClip` as a glTF/GLB asset that loads with Three.js `GLTFLoader`. Both sample the timeline headlessly (no WebGL) at a configurable frame rate and bake the full looped runtime; they export authored motion, not the contact/IK-solved motion. diff --git a/README.md b/README.md index a0942ea..4f7670a 100644 --- a/README.md +++ b/README.md @@ -409,6 +409,24 @@ npm run build --- +### Editor support + +A VS Code extension provides syntax highlighting, ROM diagnostics, and +completion for `.posecode` files — see +[`editors/vscode`](editors/vscode/README.md). Until it is published, you can +get basic highlighting immediately by associating `.posecode` files with +Markdown: + +```json +"files.associations": { + "*.posecode": "markdown" +} +``` + +See the [editor guide](editors/vscode/README.md#file-association-before-the-extension-is-installed) +for VS Code, Cursor, Sublime Text, and Neovim instructions. + +--- ## MCP Server @@ -505,6 +523,64 @@ if (!ir || errors.length > 0) { The `#viewer` element is an HTML ``. +### Exporting motion (BVH) + +`posecode-render` can bake a movement into a [Biovision Hierarchy](https://en.wikipedia.org/wiki/Biovision_Hierarchy) +(`.bvh`) file for import into Blender and other animation tools. In the +playground, use the **Download BVH** button; programmatically: + +```ts +import { parse } from "posecode-parser"; +import { exportBVH } from "posecode-render"; + +const { ir } = parse(source); +const bvh = exportBVH(ir!, { fps: 30 }); // string, ready to write to disk +``` + +Options: `fps` (default 30), `scale` (default 1 = metres; pass `100` for +centimetres), `includeFingers` (default false), and `proportions` for a +calibrated rig. + +- **Coordinate system:** right-handed, **Y-up**, figure faces **+Z** in the + rest pose (identical to the renderer and Three.js). Enable Blender's "Y up" + BVH import option. +- **Units:** metres by default. +- **Rotation channels:** `Zrotation Xrotation Yrotation` (Euler order `ZXY`). +- **Scope:** this exports the *authored* joint motion plus root travel/turn. It + does not yet re-run the renderer's contact/IK solve, so IK-dependent movements + (e.g. `reach: hand_left floor`) export the authored pose rather than the + solved one. See [issue #63](https://github.com/posecode-dev/posecode/issues/63). + +### Exporting motion (glTF / GLB) + +For web animation pipelines, `posecode-render` can export the rig **and** a +baked animation clip as a glTF/GLB asset. In the playground, use **Download +glTF**; programmatically: + +```ts +import { parse } from "posecode-parser"; +import { exportGLTF } from "posecode-render"; + +const { ir } = parse(source); +const glb = await exportGLTF(ir!); // GLB ArrayBuffer (default) +const gltf = await exportGLTF(ir!, { binary: false }); // glTF JSON object +``` + +The result loads with Three.js [`GLTFLoader`](https://threejs.org/docs/#GLTFLoader.load), +and the clip plays on the included rig: + +```ts +const gltf = await new GLTFLoader().loadAsync(url); +const mixer = new THREE.AnimationMixer(gltf.scene); +mixer.clipAction(gltf.animations[0]).play(); +``` + +- Joint nodes are named by Posecode bone id; the animated root is `posecode_root`. +- **Limitations:** exports the procedural mannequin rig, not a humanoid/Mixamo + skeleton, so there is **no retargeting** onto external rigs yet, and (as with + BVH) it bakes the authored motion, not the contact/IK-solved motion. See + [issue #90](https://github.com/posecode-dev/posecode/issues/90). + --- ## How Posecode Stays Honest diff --git a/ROADMAP.md b/ROADMAP.md index e623f5a..a14a5c1 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -122,7 +122,11 @@ Each prop is a small scene object + an anchor type; movements then reference it - Self-collision is a bounded corrective pass over selected body pairs, not a comprehensive physics system. It exposes residuals for those sampled pairs, but does not detect every possible body-body collision. -- There is no glTF/GLB or BVH motion export yet. +- BVH and glTF/GLB motion export bake the **authored** joint motion and root + choreography (travel/turn); they do not yet re-run the renderer's contact/IK + solve, so IK-dependent movements export their authored pose rather than the + solved one. glTF export uses the procedural mannequin rig with no retargeting + onto external/humanoid skeletons yet. - A **starter** prop set (chair / wall / bar / box / dip bars): no bench, rings, bands, or loaded implements yet, and props sit at fixed default placements. diff --git a/editors/vscode/README.md b/editors/vscode/README.md index ef3517e..ec5a1a4 100644 --- a/editors/vscode/README.md +++ b/editors/vscode/README.md @@ -9,6 +9,52 @@ Language support for the **Posecode** (`.posecode`) kinematic motion DSL: The smart features are provided by [`posecode-lsp`](../../packages/posecode-lsp), which shares its language logic ([`posecode-language`](../../packages/posecode-language)) with the web playground, so the editor and the playground always agree. +## File association (before the extension is installed) + +Until the full extension is published to the Marketplace, `.posecode` files +open as plain text. You can get basic highlighting and comment/bracket +behaviour right away by telling your editor to treat `.posecode` files as +Markdown, which is the closest built-in grammar. + +### VS Code + +Add the following to your `settings.json` (open the Command Palette → +**Preferences: Open User Settings (JSON)**, or use a workspace +`.vscode/settings.json` to scope it to a single project): + +```json +{ + "files.associations": { + "*.posecode": "markdown" + } +} +``` + +Alternatively, open any `.posecode` file, click the language indicator in the +bottom-right status bar (it will say "Plain Text"), choose **Configure File +Association for '.posecode'…**, and pick **Markdown**. + +### Cursor and other VS Code forks + +Cursor, VSCodium, and other VS Code forks read the same `files.associations` +setting, so the JSON snippet above works unchanged. + +### Sublime Text + +Open a `.posecode` file, then use the menu **View → Syntax → Open all with +current extension as… → Markdown**. + +### Neovim + +Register the extension in your config: + +```lua +vim.filetype.add({ extension = { posecode = "markdown" } }) +``` + +Once the dedicated extension is installed it registers the real `posecode` +language id, and you can remove these fallbacks. + ## Develop / run locally From the repo root: diff --git a/packages/posecode-render/src/bvh.ts b/packages/posecode-render/src/bvh.ts new file mode 100644 index 0000000..b33fa79 --- /dev/null +++ b/packages/posecode-render/src/bvh.ts @@ -0,0 +1,228 @@ +/** + * BVH motion export. + * + * Bakes a Posecode movement into a Biovision Hierarchy (`.bvh`) file so an + * authored movement can be imported into Blender and other animation tools. + * + * ## What is exported + * + * This exporter bakes the **authored joint motion**: the timeline's forward + * kinematics (joint rotations per phase) plus the root choreography (travel + * translation and turn yaw). It does NOT run the renderer's contact solve + * (ground-lock, reach/pin/grip IK, floor clamping), so movements whose final + * look depends on IK — e.g. a `reach: hand_left floor` — will export the + * authored pose rather than the solved one. Purely FK-authored movements + * (squats, curls, ballet port de bras, most of the library) round-trip + * faithfully. Exporting the fully solved motion is a documented future + * enhancement; see issue #63. + * + * ## Coordinate system and scale + * + * - Right-handed, **Y-up**, figure facing **+Z** in the rest pose — identical + * to the renderer's rig, and to Three.js' default. Blender's BVH importer + * has a "Y up" option; enable it (or apply a +90° X rotation on import). + * - Units are **metres** by default. Pass `scale: 100` to emit centimetres if + * your tool expects that. + * - Joint rotation channels use the `Zrotation Xrotation Yrotation` order + * (Euler order `ZXY`), the most widely compatible BVH convention. + */ + +import * as THREE from "three"; +import type { PosecodeIR } from "posecode-parser"; +import { buildMannequin, type Proportions } from "./mannequin.js"; +import { buildTimeline } from "./timeline.js"; + +/** Finger joints, excluded by default to keep the skeleton importer-friendly. */ +const FINGER_PREFIXES = ["thumb", "index", "middle", "ring", "pinky"]; + +const DEFAULT_FPS = 30; +/** BVH channel order `Zrotation Xrotation Yrotation` ⇔ Three.js Euler `ZXY`. */ +const EULER_ORDER = "ZXY" as const; + +export interface BvhExportOptions { + /** Sample rate for the baked keyframes. Defaults to 30 fps. */ + fps?: number; + /** Include the per-finger curl joints (30 extra channels). Defaults to false. */ + includeFingers?: boolean; + /** Multiply every length by this factor. 1 = metres (default), 100 = cm. */ + scale?: number; + /** Rig proportions, if exporting for a calibrated character. */ + proportions?: Proportions; +} + +function isFingerBone(id: string): boolean { + return FINGER_PREFIXES.some((p) => id.startsWith(p)); +} + +/** A joint in the export skeleton, mirroring the live mannequin bone tree. */ +interface ExportJoint { + id: string; + node: THREE.Object3D; + children: ExportJoint[]; +} + +/** + * Sensible End Site tip offset (metres, local frame) for a leaf joint, so the + * exported skeleton reads correctly in an importer. Cosmetic — any non-zero + * value produces a valid file. + */ +function endSiteOffset(id: string): [number, number, number] { + if (id === "head") return [0, 0.12, 0]; + if (id.startsWith("ankle")) return [0, -0.04, 0.14]; // toe, forward + if (isFingerBone(id)) return [0, -0.03, 0]; + return [0, -0.08, 0]; // generic distal extension +} + +/** Build the export skeleton tree from a fresh, rest-posed mannequin. */ +function buildExportSkeleton( + bones: Map, + root: THREE.Object3D, + includeFingers: boolean, +): ExportJoint { + const boneNodes = new Set(bones.values()); + const make = (id: string, node: THREE.Object3D): ExportJoint => { + const children: ExportJoint[] = []; + for (const [childId, childNode] of bones) { + if (childNode.parent !== node) continue; + if (!includeFingers && isFingerBone(childId)) continue; + children.push(make(childId, childNode)); + } + // Deterministic child order keeps output stable across runs. + children.sort((a, b) => a.id.localeCompare(b.id)); + return { id, node, children }; + }; + // The root joint is the single bone parented directly to the rig group. + for (const [id, node] of bones) { + if (node.parent === root || !boneNodes.has(node.parent as THREE.Object3D)) { + return make(id, node); + } + } + throw new Error("posecode BVH export: could not locate a root joint"); +} + +function fmt(n: number): string { + // Trim to 6 decimals, then strip trailing zeros for compact, stable output. + return Number(n.toFixed(6)).toString(); +} + +/** Enumerate joints in the exact depth-first order channel values are written. */ +function flatten(joint: ExportJoint, out: ExportJoint[]): void { + out.push(joint); + for (const child of joint.children) flatten(child, out); +} + +function writeHierarchy( + joint: ExportJoint, + scale: number, + depth: number, + isRoot: boolean, +): string { + const pad = " ".repeat(depth); + const lines: string[] = []; + if (isRoot) { + lines.push(`${pad}ROOT ${joint.id}`); + lines.push(`${pad}{`); + // The root joint carries the world translation in its position channels, + // so its own OFFSET is the origin. + lines.push(`${pad} OFFSET 0.000000 0.000000 0.000000`); + lines.push( + `${pad} CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation`, + ); + } else { + const o = joint.node.position; + lines.push(`${pad}JOINT ${joint.id}`); + lines.push(`${pad}{`); + lines.push( + `${pad} OFFSET ${fmt(o.x * scale)} ${fmt(o.y * scale)} ${fmt(o.z * scale)}`, + ); + lines.push(`${pad} CHANNELS 3 Zrotation Xrotation Yrotation`); + } + if (joint.children.length > 0) { + for (const child of joint.children) { + lines.push(writeHierarchy(child, scale, depth + 1, false)); + } + } else { + // Leaf joint: BVH requires a terminating End Site with a tip offset. + const [ex, ey, ez] = endSiteOffset(joint.id); + lines.push(`${pad} End Site`); + lines.push(`${pad} {`); + lines.push(`${pad} OFFSET ${fmt(ex * scale)} ${fmt(ey * scale)} ${fmt(ez * scale)}`); + lines.push(`${pad} }`); + } + lines.push(`${pad}}`); + return lines.join("\n"); +} + +const _euler = new THREE.Euler(); +const _yawQuat = new THREE.Quaternion(); +const _rootQuat = new THREE.Quaternion(); +const _up = new THREE.Vector3(0, 1, 0); + +/** Extract `Zrotation Xrotation Yrotation` degrees from a local quaternion. */ +function eulerChannels(q: THREE.Quaternion): [number, number, number] { + _euler.setFromQuaternion(q, EULER_ORDER); + const RAD2DEG = 180 / Math.PI; + return [_euler.z * RAD2DEG, _euler.x * RAD2DEG, _euler.y * RAD2DEG]; +} + +/** + * Export a parsed Posecode movement as BVH text. + * + * The returned string is a complete `.bvh` document (HIERARCHY + MOTION) ready + * to write to disk or hand to a browser download. + */ +export function exportBVH(ir: PosecodeIR, options: BvhExportOptions = {}): string { + const fps = options.fps && options.fps > 0 ? options.fps : DEFAULT_FPS; + const scale = options.scale && options.scale > 0 ? options.scale : 1; + const includeFingers = options.includeFingers ?? false; + + const mannequin = buildMannequin(undefined, options.proportions); + const timeline = buildTimeline(ir); + const skeleton = buildExportSkeleton( + mannequin.bones, + mannequin.root, + includeFingers, + ); + + const flat: ExportJoint[] = []; + flatten(skeleton, flat); + const rootRestY = skeleton.node.position.y; + + // Bake one frame per 1/fps across the full played duration (all repeats), so + // the loop count and total runtime survive the export as literal keyframes. + const cycle = Math.max(timeline.duration, 1e-6); + const total = cycle * Math.max(1, timeline.repeat); + const dt = 1 / fps; + const frameCount = Math.max(1, Math.round(total * fps)) + 1; + + const motionRows: string[] = []; + for (let f = 0; f < frameCount; f++) { + const t = Math.min(f * dt, total); + const info = timeline.sample(t, mannequin.bones); + + // Root world orientation folds the body yaw into the pelvis local rotation + // (the root joint has no parent, so its channels are world-space). + _yawQuat.setFromAxisAngle(_up, info.rootYaw); + _rootQuat.copy(_yawQuat).multiply(skeleton.node.quaternion); + + const row: number[] = []; + for (const joint of flat) { + if (joint === skeleton) { + row.push(info.rootOffset.x * scale, rootRestY * scale, info.rootOffset.z * scale); + row.push(...eulerChannels(_rootQuat)); + } else { + row.push(...eulerChannels(joint.node.quaternion)); + } + } + motionRows.push(row.map(fmt).join(" ")); + } + + const header = [ + "HIERARCHY", + writeHierarchy(skeleton, scale, 0, true), + "MOTION", + `Frames: ${frameCount}`, + `Frame Time: ${fmt(dt)}`, + ]; + return `${header.join("\n")}\n${motionRows.join("\n")}\n`; +} diff --git a/packages/posecode-render/src/gltf.ts b/packages/posecode-render/src/gltf.ts new file mode 100644 index 0000000..b8ac563 --- /dev/null +++ b/packages/posecode-render/src/gltf.ts @@ -0,0 +1,150 @@ +/** + * glTF / GLB animation export. + * + * Bakes a Posecode movement into a standard glTF asset — the rig plus a baked + * animation clip — so it can drop into an existing web animation pipeline and + * load with Three.js' `GLTFLoader`. + * + * ## What is exported + * + * The procedural mannequin rig (bone hierarchy with limb meshes parented to + * each joint) and one `AnimationClip` that drives the joint rotations and the + * root travel/turn. Like the BVH path, this bakes the **authored** joint motion + * plus root choreography; it does not re-run the renderer's contact/IK solve, + * so IK-dependent movements export their authored pose. Exporting the fully + * solved motion, and retargeting onto external/humanoid skeletons, are + * documented future work (see issue #90). + * + * ## Conventions + * + * - Right-handed, **Y-up**, figure faces **+Z** at rest (Three.js default). + * - Units are metres. + * - Joint nodes are named by their Posecode bone id (`elbow_left`, `hip_right`, + * …); the animated root group is `posecode_root`. + * - The full looped runtime (cycle incl. loop-reset wrap × repeats) is baked as + * keyframes, so duration and loop count survive without a runtime loop flag. + */ + +import * as THREE from "three"; +import { GLTFExporter } from "three/examples/jsm/exporters/GLTFExporter.js"; +import type { PosecodeIR } from "posecode-parser"; +import { buildMannequin, type Proportions } from "./mannequin.js"; +import { buildTimeline } from "./timeline.js"; + +const DEFAULT_FPS = 30; +const ROOT_NODE_NAME = "posecode_root"; + +export interface GltfExportOptions { + /** Keyframe sample rate. Defaults to 30 fps. */ + fps?: number; + /** true → GLB binary ArrayBuffer (default); false → glTF JSON object. */ + binary?: boolean; + /** Rig proportions, if exporting for a calibrated character. */ + proportions?: Proportions; +} + +const _up = new THREE.Vector3(0, 1, 0); +const _yaw = new THREE.Quaternion(); + +/** + * Build the mannequin rig and a baked `AnimationClip` for a movement, without + * touching the DOM. Exposed for tests and advanced callers; most consumers want + * {@link exportGLTF}. + */ +export function buildAnimatedRig( + ir: PosecodeIR, + options: GltfExportOptions = {}, +): { root: THREE.Group; clip: THREE.AnimationClip } { + const fps = options.fps && options.fps > 0 ? options.fps : DEFAULT_FPS; + const mannequin = buildMannequin(undefined, options.proportions); + const timeline = buildTimeline(ir); + + // Name every joint node by its bone id so animation tracks bind by name and + // the exported glTF uses a stable, documented joint-naming convention. + mannequin.root.name = ROOT_NODE_NAME; + const animatedBones: string[] = []; + for (const [id, node] of mannequin.bones) { + node.name = id; + animatedBones.push(id); + } + + const cycle = Math.max(timeline.duration, 1e-6); + const total = cycle * Math.max(1, timeline.repeat); + const dt = 1 / fps; + const frameCount = Math.max(1, Math.round(total * fps)) + 1; + + const times = new Float32Array(frameCount); + const rootPos = new Float32Array(frameCount * 3); + const rootQuat = new Float32Array(frameCount * 4); + const boneQuat = new Map(); + for (const id of animatedBones) boneQuat.set(id, new Float32Array(frameCount * 4)); + + for (let f = 0; f < frameCount; f++) { + const t = Math.min(f * dt, total); + times[f] = t; + const info = timeline.sample(t, mannequin.bones); + + // Root group carries the world travel (x,z) and the body yaw. + rootPos[f * 3] = info.rootOffset.x; + rootPos[f * 3 + 1] = 0; + rootPos[f * 3 + 2] = info.rootOffset.z; + _yaw.setFromAxisAngle(_up, info.rootYaw); + rootQuat[f * 4] = _yaw.x; + rootQuat[f * 4 + 1] = _yaw.y; + rootQuat[f * 4 + 2] = _yaw.z; + rootQuat[f * 4 + 3] = _yaw.w; + + for (const id of animatedBones) { + const q = mannequin.bones.get(id)!.quaternion; + const buf = boneQuat.get(id)!; + buf[f * 4] = q.x; + buf[f * 4 + 1] = q.y; + buf[f * 4 + 2] = q.z; + buf[f * 4 + 3] = q.w; + } + } + + const tracks: THREE.KeyframeTrack[] = [ + new THREE.VectorKeyframeTrack(`${ROOT_NODE_NAME}.position`, Array.from(times), Array.from(rootPos)), + new THREE.QuaternionKeyframeTrack(`${ROOT_NODE_NAME}.quaternion`, Array.from(times), Array.from(rootQuat)), + ]; + for (const id of animatedBones) { + tracks.push( + new THREE.QuaternionKeyframeTrack( + `${id}.quaternion`, + Array.from(times), + Array.from(boneQuat.get(id)!), + ), + ); + } + + const clip = new THREE.AnimationClip(ir.name || "posecode", total, tracks); + // Reset the rig to its rest pose so the exported node transforms are neutral; + // the clip supplies all motion. + timeline.sample(0, mannequin.bones); + return { root: mannequin.root, clip }; +} + +/** + * Export a parsed Posecode movement as a glTF/GLB asset. + * + * Returns a GLB `ArrayBuffer` (default) or a glTF JSON object when + * `binary: false`. The result loads with Three.js `GLTFLoader`, and the baked + * clip plays on the included rig. + */ +export async function exportGLTF( + ir: PosecodeIR, + options: GltfExportOptions = {}, +): Promise> { + const { root, clip } = buildAnimatedRig(ir, options); + const exporter = new GLTFExporter(); + const binary = options.binary ?? true; + return await new Promise((resolve, reject) => { + exporter.parse( + root, + (result) => resolve(result as ArrayBuffer | Record), + (error) => reject(error), + { binary, animations: [clip] }, + ); + }); +} diff --git a/packages/posecode-render/src/index.ts b/packages/posecode-render/src/index.ts index c399491..fbc484b 100644 --- a/packages/posecode-render/src/index.ts +++ b/packages/posecode-render/src/index.ts @@ -1546,3 +1546,5 @@ export { type FootContactMeasurement, } from "./contacts.js"; export type { PhaseSegment } from "./timeline.js"; +export { exportBVH, type BvhExportOptions } from "./bvh.js"; +export { exportGLTF, buildAnimatedRig, type GltfExportOptions } from "./gltf.js"; diff --git a/packages/posecode-render/test/bvh.test.ts b/packages/posecode-render/test/bvh.test.ts new file mode 100644 index 0000000..b7b36b7 --- /dev/null +++ b/packages/posecode-render/test/bvh.test.ts @@ -0,0 +1,169 @@ +import { describe, expect, it } from "vitest"; +import * as THREE from "three"; +import { parse } from "posecode-parser"; +import { buildMannequin } from "../src/mannequin.js"; +import { buildTimeline } from "../src/timeline.js"; +import { exportBVH } from "../src/bvh.js"; + +const BICEPS = `posecode exercise "Biceps curl" + rig humanoid + pose start = standing + + step "Curl" 1.1s settle: + elbows: flex 135 + step "Lower" 1.4s settle: + elbows: flex 15 + repeat 3 +`; + +const GRAPEVINE = `posecode exercise "Grapevine" + rig humanoid + pose start = standing + + step "Left step side" 0.65s flow: + hip_left: abduct 16 + travel: 0.3 0 + step "Return" 0.65s flow: + hip_left: abduct 0 + travel: 0 0 + repeat 1 +`; + +/** Minimal BVH reader: joint names in DFS order + channel spans for a frame. */ +interface ParsedBvh { + joints: { name: string; channels: string[]; offset: number }[]; + frames: number[][]; + frameTime: number; + channelCount: number; +} + +function readBvh(text: string): ParsedBvh { + const lines = text.split("\n"); + const joints: ParsedBvh["joints"] = []; + let channelCount = 0; + let i = 0; + let current: string | null = null; + for (; i < lines.length; i++) { + const line = lines[i]!.trim(); + if (line === "MOTION") break; + const jointMatch = /^(ROOT|JOINT)\s+(\S+)/.exec(line); + if (jointMatch) current = jointMatch[2]!; + const chanMatch = /^CHANNELS\s+(\d+)\s+(.*)$/.exec(line); + if (chanMatch && current) { + const channels = chanMatch[2]!.trim().split(/\s+/); + joints.push({ name: current, channels, offset: channelCount }); + channelCount += channels.length; + } + } + // MOTION block. + const framesLine = lines[++i]!.trim(); + const frameCount = Number(/Frames:\s+(\d+)/.exec(framesLine)![1]); + const frameTime = Number(/Frame Time:\s+([\d.eE+-]+)/.exec(lines[++i]!.trim())![1]); + const frames: number[][] = []; + for (let f = 0; f < frameCount; f++) { + frames.push(lines[++i]!.trim().split(/\s+/).map(Number)); + } + return { joints, frames, frameTime, channelCount }; +} + +/** Reconstruct a joint's local quaternion from its Z/X/Y rotation channels. */ +function quatFromChannels( + joint: ParsedBvh["joints"][number], + row: number[], +): THREE.Quaternion { + const DEG2RAD = Math.PI / 180; + let x = 0; + let y = 0; + let z = 0; + joint.channels.forEach((chan, idx) => { + const v = row[joint.offset + idx]! * DEG2RAD; + if (chan === "Xrotation") x = v; + if (chan === "Yrotation") y = v; + if (chan === "Zrotation") z = v; + }); + return new THREE.Quaternion().setFromEuler(new THREE.Euler(x, y, z, "ZXY")); +} + +describe("exportBVH", () => { + it("emits a well-formed hierarchy and matching frame data", () => { + const { ir } = parse(BICEPS); + expect(ir).toBeTruthy(); + const bvh = exportBVH(ir!, { fps: 30 }); + + expect(bvh.startsWith("HIERARCHY")).toBe(true); + expect(bvh).toContain("ROOT pelvis"); + expect(bvh).toContain("MOTION"); + // Root has 6 channels, and no fingers by default. + expect(bvh).toContain( + "CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation", + ); + expect(bvh).not.toContain("thumb_left"); + + const parsed = readBvh(bvh); + // pelvis root + 4 spine/head + 6 arm + 6 leg = 17 joints, 6 + 16*3 = 54 channels. + expect(parsed.joints[0]!.name).toBe("pelvis"); + expect(parsed.joints[0]!.channels).toHaveLength(6); + // Every motion row carries exactly the declared channel count. + for (const row of parsed.frames) { + expect(row).toHaveLength(parsed.channelCount); + } + // The whole looped runtime (cycle incl. loop-reset wrap × reps) is baked, + // one frame per 1/fps plus the closing frame. + const tl = buildTimeline(ir!); + const total = tl.duration * tl.repeat; + expect(parsed.frameTime).toBeCloseTo(1 / 30, 6); + expect(parsed.frames.length).toBe(Math.round(total * 30) + 1); + }); + + it("bakes joint rotations that reconstruct the sampled pose", () => { + const { ir } = parse(BICEPS); + const bvh = exportBVH(ir!, { fps: 30 }); + const parsed = readBvh(bvh); + const elbow = parsed.joints.find((j) => j.name === "elbow_left")!; + expect(elbow).toBeTruthy(); + + // Independently sample the timeline at the same frame time and compare the + // reconstructed elbow rotation against the live bone rotation. + const mannequin = buildMannequin(); + const timeline = buildTimeline(ir!); + const frameIndex = 15; // 0.5s in, mid-curl + timeline.sample(frameIndex / 30, mannequin.bones); + const expected = mannequin.bones.get("elbow_left")!.quaternion; + + const got = quatFromChannels(elbow, parsed.frames[frameIndex]!); + expect(got.angleTo(expected)).toBeLessThan(1e-4); + + // The curl is a real motion: the elbow is clearly bent at mid-curl. + expect(Math.abs(new THREE.Euler().setFromQuaternion(expected, "ZXY").x)).toBeGreaterThan(0.5); + }); + + it("preserves root translation for a travelling movement", () => { + const { ir } = parse(GRAPEVINE); + const bvh = exportBVH(ir!, { fps: 30 }); + const parsed = readBvh(bvh); + // Root position lives in the first three channels (Xposition Yposition Zposition). + const xs = parsed.frames.map((row) => row[0]!); + const ys = parsed.frames.map((row) => row[1]!); + // Travels +0.3m in X then back: peak X well above the start. + expect(Math.max(...xs)).toBeGreaterThan(0.2); + expect(xs[0]).toBeCloseTo(0, 3); + // The pelvis height stays constant in authored export (no ground solve). + expect(Math.max(...ys) - Math.min(...ys)).toBeCloseTo(0, 6); + }); + + it("can include finger joints on request", () => { + const { ir } = parse(BICEPS); + const withFingers = readBvh(exportBVH(ir!, { includeFingers: true })); + const without = readBvh(exportBVH(ir!, { includeFingers: false })); + expect(withFingers.joints.length).toBeGreaterThan(without.joints.length); + expect(withFingers.joints.some((j) => j.name === "thumb_left")).toBe(true); + }); + + it("scales lengths when asked (metres → centimetres)", () => { + const { ir } = parse(GRAPEVINE); + const cm = readBvh(exportBVH(ir!, { scale: 100 })); + const peakX = Math.max(...cm.frames.map((row) => row[0]!)); + // 0.3 m travel becomes ~30 cm. + expect(peakX).toBeGreaterThan(20); + }); +}); diff --git a/packages/posecode-render/test/gltf.test.ts b/packages/posecode-render/test/gltf.test.ts new file mode 100644 index 0000000..0cfe79a --- /dev/null +++ b/packages/posecode-render/test/gltf.test.ts @@ -0,0 +1,113 @@ +import { describe, expect, it, beforeAll } from "vitest"; +import * as THREE from "three"; +import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader.js"; +import { parse } from "posecode-parser"; +import { exportGLTF, buildAnimatedRig } from "../src/gltf.js"; + +// GLTFExporter serializes buffers via FileReader, which browsers provide but +// Node does not. Polyfill it faithfully over Node's global Blob so the headless +// round-trip mirrors the browser export the playground actually runs. +beforeAll(() => { + if (typeof (globalThis as { FileReader?: unknown }).FileReader !== "undefined") return; + class NodeFileReader { + result: ArrayBuffer | string | null = null; + onloadend: (() => void) | null = null; + onerror: ((err: unknown) => void) | null = null; + readAsArrayBuffer(blob: Blob): void { + blob.arrayBuffer().then( + (buf) => { + this.result = buf; + this.onloadend?.(); + }, + (err) => this.onerror?.(err), + ); + } + readAsDataURL(blob: Blob): void { + blob.arrayBuffer().then( + (buf) => { + const b64 = Buffer.from(buf).toString("base64"); + this.result = `data:${blob.type || "application/octet-stream"};base64,${b64}`; + this.onloadend?.(); + }, + (err) => this.onerror?.(err), + ); + } + } + (globalThis as { FileReader?: unknown }).FileReader = NodeFileReader; +}); + +const BICEPS = `posecode exercise "Biceps curl" + rig humanoid + pose start = standing + + step "Curl" 1.1s settle: + elbows: flex 135 + step "Lower" 1.4s settle: + elbows: flex 15 + repeat 2 +`; + +function loadGlb(buffer: ArrayBuffer): Promise { + return new Promise((resolve, reject) => { + new GLTFLoader().parse( + buffer, + "", + (gltf) => resolve(Object.assign(gltf.scene, { animations: gltf.animations })), + reject, + ); + }); +} + +describe("exportGLTF", () => { + it("builds a rig plus a baked clip with the expected tracks", () => { + const { ir } = parse(BICEPS); + expect(ir).toBeTruthy(); + const { root, clip } = buildAnimatedRig(ir!, { fps: 30 }); + + expect(root.name).toBe("posecode_root"); + // Root gets a position + quaternion track; each joint gets a quaternion track. + expect(clip.tracks.some((t) => t.name === "posecode_root.position")).toBe(true); + expect(clip.tracks.some((t) => t.name === "elbow_left.quaternion")).toBe(true); + expect(clip.duration).toBeGreaterThan(0); + // The elbow actually moves: some frame differs from the first keyframe. + const elbow = clip.tracks.find((t) => t.name === "elbow_left.quaternion")!; + const vals = elbow.values; + const frames = vals.length / 4; + let moved = false; + for (let i = 1; i < frames && !moved; i++) { + for (let k = 0; k < 4; k++) { + if (Math.abs(vals[i * 4 + k]! - vals[k]!) > 1e-3) moved = true; + } + } + expect(moved).toBe(true); + }); + + it("exports a GLB that reloads through GLTFLoader with its animation", async () => { + const { ir } = parse(BICEPS); + const glb = await exportGLTF(ir!, { fps: 24, binary: true }); + expect(glb).toBeInstanceOf(ArrayBuffer); + + const scene = await loadGlb(glb as ArrayBuffer); + // The rig survived the round-trip. + expect(scene.getObjectByName("elbow_left")).toBeTruthy(); + // Exactly one baked animation clip, and it drives the elbow joint. + expect(scene.animations).toHaveLength(1); + const clip = scene.animations[0]!; + expect(clip.duration).toBeGreaterThan(0); + expect(clip.tracks.some((t) => /elbow_left\.quaternion$/.test(t.name))).toBe(true); + + // The clip can be bound to a mixer and advanced without error (plays). + const mixer = new THREE.AnimationMixer(scene); + const action = mixer.clipAction(clip); + action.play(); + expect(() => mixer.update(0.5)).not.toThrow(); + }); + + it("can emit a glTF JSON object instead of GLB", async () => { + const { ir } = parse(BICEPS); + const gltf = (await exportGLTF(ir!, { binary: false })) as Record; + expect(gltf).toMatchObject({ asset: expect.anything() }); + expect(Array.isArray(gltf.animations)).toBe(true); + expect((gltf.animations as unknown[]).length).toBe(1); + }); +}); diff --git a/playground/play.html b/playground/play.html index 0d04184..87479a4 100644 --- a/playground/play.html +++ b/playground/play.html @@ -119,6 +119,22 @@ Share + +