This document describes what the firmware does. History, prior art, speculative modality ideas, and philosophy live in research.md. Section numbers are kept stable, so there is a gap (2).
| Layer | Fact |
|---|---|
| Implemented | 20 Hz loop: sense → physiology → drives → memory → action → actuate → learn. Learned sensor→actuator reflexes trained by drive reduction. Named-channel body. SoftAP dashboard. Life-state JSON. Station firmware as a separate state machine. |
| Tested | Host-native tests for bit packing, age saturation, rate-limit clamp, LEDC cap, and organism behavior (energy dynamics, contingency bias, reflex learning rule) against a fake body. Not real hardware. |
| Simulated | The unmodified engine in a modelled room with a station (src/sim): learned reflexes lift mean lifespan from 1.26 h to 1.86 h (hand-wired bound 2.46 h). Learns to reach and stay on the dock; not yet to leave when full, and nothing accumulates across lives. |
| Hardware tested | Nothing. No board has run this. |
| Designed / planned | Body declared in a JSON file with per-input ranges, stepper and mic-envelope channel kinds (example build); approach-and-dock on hardware; extra boards. |
| Hypothesis | That drives + decaying memory + learned reflexes + noise will look alive on real hardware, including charging as an attractor. Unverified. |
The problem is simple to state and hard to do well: make a small, cheap, battery-powered machine look alive.
Not intelligent in the large-model sense, not conversational, and not necessarily useful for a fixed task. Instead, we want a machine that appears to have:
Humans are extremely sensitive to motion, timing, and contingency. A few sensors, motors, delays, and feedback loops are enough for us to describe behavior using words like curious, cautious, bored, or interested. Heider and Simmel’s classic experiment showed that people attribute agency and intent even to moving geometric shapes.
In this project, such words are observer-level summaries of dynamics, not claims about subjective experience. The concrete engineering goal is:
Construct a machine whose behavior is best understood in terms of internal regulation, learning, and interaction with its environment, rather than in terms of fixed scripts.
The central constraints are deliberate:
Instead, we rely on:
The remainder of this document presents the system architecture, algorithms, and experimental considerations for this synthetic ethology platform. Theory, prior art, and philosophical framing live separately in research.md.
We aim for an architecture that:
Runs on an ESP32-class microcontroller.
Treats signals symmetrically.
Implements a minimal “organism” architecture.
Connects behavior to energy without scripting it.
h_energy. Supports serializable “life states”.
Is suitable as a research/education platform.
Everything above the gateway is the behavior engine, header-only and identical in the
firmware and the simulator. The only reward anywhere is the drop
in drive pressure; the only privileged wire is battery → h_energy.
Body layer
Core loop
Observation layer
Actuators are generic effectors. The system does not know what they “mean”.
Example actuator set:
| Name | Type | Range | Physical Example |
|---|---|---|---|
motor_left/right |
continuous | -1 … +1 | Differential drive motors |
servo_pan/tilt |
continuous | -1 … +1 | Head or sensor orientation |
vibe_motor |
continuous | 0 … 1 | Vibration motor |
grind_motor |
continuous | 0 … 1 | Noisy mechanical actuator (“voice”) |
led_r/g/b |
continuous | 0 … 1 | RGB lighting / signaling |
fan |
continuous | 0 … 1 | Airflow actuator |
pump/atomizer |
continuous | 0 … 1 | Fluid or scent emission |
heater |
continuous | 0 … 1 | Local thermal field |
electromagnet |
continuous | 0 … 1 | Gripping, field perturbation |
The reference boards (src/esp32/include/boards/) wire only motor_left/right,
led_status, led_r/g/b, and head_pan. Implemented actuator kinds:
kPwmUnipolar, kPwmBidirectional, kDigitalOut, kServo.
Each actuator has:
Sensors provide normalized numeric streams. No label is privileged.
Example sensors:
| Name | Type | Example Hardware |
|---|---|---|
mic_energy, bands, peaks |
continuous | MEMS microphone |
distance_* |
continuous | ToF, ultrasonic, IR |
touch_* |
cont/binary | Bump switches, capacitive strips |
light_*, uv |
continuous | Photodiodes, light sensors |
imu_accel/gyro/mag |
continuous | 6-axis or 9-axis IMU |
temperature, humidity |
continuous | Environmental sensors |
gas, air_quality |
continuous | Metal-oxide gas sensor |
rf_rssi_* |
continuous | Wi-Fi/BLE RSSI or beacons |
battery_voltage, current |
continuous | ADC monitoring of power rails |
internal_temp |
continuous | MCU or board temperature |
The reference boards wire only battery_voltage, touch_bump, and
rf_rssi_station. Implemented sensor kinds: kAdcNormalized, kDigitalIn,
kWifiRssi.
Battery signals enter at this layer as raw numbers, like any other sensor.
We do not predefine kinematics or a body model. The intent (untested on hardware) is that contingency memory picks up regularities such as:
motor_left and motor_right change like this, IMU and distance_front change like that.” servo_pan increases, a particular light sensor often saturates.”This echoes how infants learn their own bodies through co-occurrence of actuation, proprioception, and exteroception.
The internal state contains a small vector of slow variables, for example:
h_energy: resource level. With an energy sensor (the battery) it simply follows that reading; the battery already reflects exertion, so nothing else drains it. Without one, a modelled drain (actuator cost) and trickle regen stand in. h_fatigue: saturating exertion, dF = g·cost·(1−F) − r·F. Settles near 0.3 at moderate effort; reaches the tired band only under sustained high effort. h_safety: drops on startle, a salient jolt on any channel, which habituates when the same channel keeps jolting; recovers slowly. h_arousal: rises with salient change, decays. Only over-arousal is a drive. h_curiosity: rises with contingency memory's prediction error. h_boredom: habituation. Rises whenever less happens than usually does here, and is reset by more-than-usual change.Salience. Every sensor channel is predicted to keep changing at the rate it just did, and only a miss beyond a few times that channel's own learned jitter counts as an event. So ADC noise is never an event, a steady turn habituates away, and a bump, a new light or a sound stands out.
A social variable (history of correlated signals from other robots/humans) was considered but isn't implemented: it would need a "presence of another agent" sensor no current board has, and an unimplemented drive that never moves is worse than no drive at all. See research notes for the idea.
Each variable:
Drives are scalar pressures derived from deviations from preferred bands. The bands
are part of the organism's constitution: energy [0.65, 1] (hunger from about half
charge on a LiPo read across its full voltage span), fatigue [0, 0.6], safety
[0.7, 1], arousal [0, 0.7], boredom [0, 0.4].
drive[name] = deviation(h_name, target_range[name])
homeostatic distance D = sum over drives except curiosity of drive^2
reward each tick = D(t-1) - D(t)
That reward, drive reduction, is the only one anything learns from (Keramati & Gutkin 2014, with their exponent n = 2: relieving a large deficit outweighs small fluctuations). Curiosity is left out of it on purpose: it rises with surprise, so counting it would make every novel outcome a setback.
Drives shape the scale and direction of stochastic exploration.
def compute_drives(phys, targets):
drives = {}
for name, value in phys.items():
lo, hi = targets[name]
if value < lo:
drives[name] = (lo - value) / (hi - lo + 1e-6)
elif value > hi:
drives[name] = (value - hi) / (hi - lo + 1e-6)
else:
drives[name] = 0.0
return drives
def calculate_fuzz_scale(drives):
curiosity = drives.get("h_curiosity", 0.0)
boredom = drives.get("h_boredom", 0.0)
energy = drives.get("h_energy", 0.0) # high = low resource
safety = drives.get("h_safety", 0.0) # high = unsafe
fuzz = (
0.3 * curiosity +
0.4 * boredom +
0.3 * energy -
0.3 * safety
)
return clamp(fuzz, 0.05, 0.8)
Intuitively:
Exploiting what has been learned when hungry comes from the hunger-gated reflexes (§12), not from standing still.
Contingency memory holds many small records of “when I did X, Y tended to follow”:
Entry:
action_delta_code # compressed Δaction at t-1
sensor_delta_code # compressed Δsensor at t
strength # reliability / confidence
age # ticks since last reinforcement
ctx_hash # 1 bit per physiology variable, set if >= 0.5
mean_drive_delta # EMA of how much total drive fell after this pattern
Quantization and hashing compress high-dimensional deltas into small codes so memory remains bounded.
LEARN = 0.1 # kTuning.contingency.learn_rate
DECAY = 0.005 # kTuning.contingency.decay_rate
PRUNE = 0.02 # kTuning.contingency.prune_threshold
MAX_ENTRIES = 256 # ContingencyMemory::kCapacity
def update_contingency(mem, d_action, d_sensor, ctx_hash):
# decay existing entries; reclaim ones that decayed to nothing
for e in mem.values():
e.strength *= (1.0 - DECAY)
e.age = min(e.age + 1, AGE_MAX)
if e.strength < PRUNE:
del mem[e.key]
key = hash_quantized(d_action, d_sensor, ctx_hash)
if key in mem:
e = mem[key]
e.strength += LEARN * (1.0 - e.strength)
e.age = 0
else:
mem[key] = Entry(
action_delta_code = compress(d_action),
sensor_delta_code = compress(d_sensor),
strength = LEARN,
age = 0,
ctx_hash = ctx_hash,
)
if len(mem) > MAX_ENTRIES:
evict_weakest(mem)
When generating new actions, the system queries for entries whose sensor_delta_code is within a small Hamming distance of what it just observed and whose mean_drive_delta is positive (the pattern helped), and biases toward the best one's action_delta_code. Patterns that only made drives worse are never recalled.
The firmware never encodes:
Those are interpretations we may assign later. For the agent, there are only:
No supported board has a positioning sensor: no encoders, no IMU, no GPS. So a "place" is a sensor signature, and spatial memory cannot steer. It only changes how much exploration noise the action generator injects.
Each sensor reading in [0,1] is binned into its quartile (2 bits per channel). The packed bits are the place's signature: two spots that read the same are the same place, and one spot whose readings drift is several places. With the reference boards' three sensors there are at most 64 distinct places.
A table of 32 cells (open addressing, evict least-visited) stores, per place:
signature,visit_count, and age (ticks since last visit, saturating),drive_improvement[var] — EMA (rate 0.15) of how much each drive fell
per tick while here.def spatial_nudge(phys, spatial, body):
best = max(spatial.cells, key=lambda c: sum(phys.drive[v] * c.drive_improvement[v] for v in VARS))
if best is None or best.score <= 0:
return 0.0
m = clamp(best.score * SPATIAL_GAIN, 0.0, SPATIAL_NUDGE_MAX) # 0.5, 0.3
return -m if best.signature == signature(body) else +m
# in the action generator, per actuator:
noise_term = noise() * fuzz * span * NOISE_GAIN * (1 + spatial_nudge)
If the current place is the best one remembered for what's pressing now, noise narrows (stay). If a better place is remembered, noise widens (move on). It does not know which way that place is.
One pass every 50 ms (20 Hz), inside Arduino loop(). CreditReflexes and
ProposeAction both happen inside ActionGenerator::tick(): the reflexes are first
credited with how drives changed since the last action, then propose the next one.
Mirrors loop() in src/esp32/src/main.cpp:
void behavior_tick(float dt_s, uint32_t now_ms) {
// 1. Physiology + drives. Reads the energy sensor and actuator costs;
// curiosity consumes contingency memory's prediction error from the
// previous tick.
phys.update(body, dt_s, contingency.last_surprise());
// 2. Learn. Contingency: (Δactuators since last tick, Δsensors since
// last tick, physiology context bits) -> reinforce/insert, tagged
// with how total drive changed. The previous tick's action is judged
// by what the sensors did over the following 50 ms; there is no
// separate propagation delay.
contingency.update(body, phys, dt_s);
spatial.update(body, phys, dt_s); // place signature + drive change here
// 3. Act. For each actuator not under manual override:
// next = clamp((reflex + explore) * (1 - rest) + contingency_bias)
// reflex: learned sensorimotor map (§12), learns from this tick's reward first
// explore: Ornstein-Uhlenbeck noise (tau 1 s) * fuzz * span * (1 + spatial_nudge)
// rest: shrink toward 0 (zero cost) with fatigue and hunger
action_gen.tick(body, phys, contingency, spatial, now_ms);
// 4. Hard limits, last: per-channel thermal budget, critical-battery cut.
safety.enforce(body, dt_s);
}
Inputs: battery_voltage, current.
No explicit “charge” routine.
Battery signals enter as normal sensor channels. The only “privilege” they have is in the physiology update:
h_energy is updated partly from battery readings and partly from behavior-dependent load estimates, h_energy is then treated like any other internal variable; drives are deviations from a comfortable band, battery_voltage are stored in contingency memory with a positive mean_drive_delta, so they can be recalled as bias when h_energy is low.Whether that adds up to something an observer would call self-charging is the hypothesis in §11.7, not a result. Other modality ideas (phototaxis, scent trails, acoustic signaling) are in research.md.
The charging station is part of the ecology, not a coded “home base”. It should:
From the robot’s perspective, the station is just a cluster of regularities:
battery_voltage, The hypothesis (§11.7) is that those regularities are enough for the robot to end up at the station when energy is low. The station firmware in src/station implements the RF beacon, LED, and click cues; nothing has been observed on hardware.
The station is:
For the robot, this appears as:
battery_voltage once contact is made.Goals:
Suggested features:
Funnel entrance:
Contact design:
Anti-snag geometry:
Thus, random or slightly biased approaches often result in a valid power connection. Once connected, geometry tends to stabilize pose and reduce unwanted motion.
The station should emit multi-modal gradients that:
Robot side: RF is just rf_rssi_*. If “being near high RSSI with that pattern” tends to precede h_energy improvement, contingencies will favor moving into RF peaks.
Robot side: appears in light_* sensors; spatial memory may encode “cells with this light distribution tend to be good for energy”.
Robot side: shows up as specific spectral signatures in microphone and/or IMU.
Robot side: changes in gas, humidity, temperature co-vary with other station cues.
Station-side power module:
Robot-side:
battery_voltage and possibly current rising, h_energy and h_safety updates.There is no explicit “start charging” RPC. Interaction is purely physical and sensory.
The station should change its sensory profile when:
These differences must be clear in sensor space but not labeled in code.
Examples:
Active charging:
Charging complete:
From the robot’s perspective:
battery_voltage correlate with improved h_energy, battery_voltage plateaus and h_energy is near target, cues stabilize into the “full” profile.Over many episodes, contingency and spatial memory can learn that:
h_energy improvement, Behavior then tends toward:
h_energy drive is low.No code ever says if low_battery then go_to_station().
src/station is real, running code, not pseudocode. Its state machine
(src/station/src/main.cpp):
charging_active = current > CURRENT_ACTIVE_THRESHOLD
if charging_active:
state = "FULL" if (voltage > VOLTAGE_FULL_THRESHOLD and
current < CURRENT_TAPER_THRESHOLD) else "CHARGING"
else:
state = "IDLE"
set_led_mode(state) # slow pulse / fast pulse / solid
set_audio_mode(state) # silent / periodic click / rare tick
# RF beacon is a fixed SoftAP SSID — it does NOT change with state.
# Near-field cues (LED, audio) carry state; RF only says "over here",
# a long-range gradient the robot can climb from a distance it can't
# yet see or hear the near-field cues from.
The robot never sees "CHARGING" or "FULL" as symbols; it only experiences
the sensory shifts (LED/audio pattern up close, RF signal strength from a
distance).
The design intent, unverified on real hardware (tracked as v0.9.1):
battery_voltage rises; h_energy improves.h_energy improvement; spatial memory notes the
location.h_energy deviates, drives bias exploration
toward patterns that historically improved it.h_energy drive relaxes, and exploration reasserts itself.If this works, an observer would describe the robot as "seeking the charger when low, leaving when full". On hardware that's still a claim about what the system is supposed to produce. In simulation (§15), steps 1–3 happen in some lives: the learned organism reaches the dock and stays. Step 4 does not: it stays docked when full. See research.md for what the hardware experiment would need to show.
A small linear map from sensor channels to actuator channels, trained only by drive
reduction (include/core/sensorimotor_policy.h). Nothing is pre-wired. If a light
sensor on one side comes to drive the opposite wheel, it's because that wiring kept
being followed by drive pressure falling: the organism built itself a Braitenberg
vehicle.
e[k][i][j] <- λ·e + (1-λ)·gate_k·x_i·ξ_j ξ = exploration sample on actuator j
w[k][i][j] += η·A·e A = reward normalized by its running mean and sd
w *= (1 - decay·dt), |w| <= 1.5 unrewarded habits fade over ~1.5 h
Memory: 7 contexts × 17 inputs × 16 outputs × 2 floats, about 15 KB at the maximum channel counts. Weights are not yet part of the saved life-state.
What it has produced (simulator only, §15): from all-zero weights, backing away after bumps (the safety context learns reverse) and, in some lives, reaching and staying on the dock. Before sensory adaptation was added it also found a stereotypy: spinning in place, which kept the light sensors changing and so relieved boredom, like pacing in under-stimulated captive animals.
The organism's state is a downloadable/restorable JSON file (GET/POST /api/state), matching what src/esp32/src/net/dashboard_server.cpp
actually serializes:
{
"version": 1,
"age_ms": 172800000,
"channel_fingerprint": "a1b2c3d4",
"physiology": { "h_energy": 0.73, "h_safety": 0.91, "...": "one entry per PhysVar" },
"contingency": [
{ "actuators": { "motor_left": 1 }, "sensors": { "light": 1 },
"ctx_hash": 12, "strength": 0.8, "mean_drive_delta": 0.05, "age": 40 }
],
"spatial": [
{ "sensors": { "light": 2 }, "visit_count": 6, "age": 3,
"drive_improvement": { "h_energy": 0.12 } }
]
}
Mechanism:
channel_fingerprint is informational only (a hash of the channel layout)
— restore is name-based regardless of whether it matches.One loop(), no FreeRTOS tasks, no interrupts:
Sensor::read(), cached per update_rate_hz — neither needs a separate
task.loop(), gated to ~20 Hz by comparing elapsed
millis() against a threshold, not a fixed-rate scheduler.StateLogger, ~1 Hz) is the same pattern at a longer
interval, inside the same loop().To stay microcontroller-friendly:
float throughout the behavior tick — deliberately, not
fixed-point: at 20 Hz with these table sizes, float arithmetic isn't the
bottleneck, and fixed-point would cost real code complexity for no
measured benefit, Run in simulation only (src/sim, which drives the unmodified engine in a modelled
room; results and the model in its README). On hardware,
none of this has been run. Currently measured: lifespan, time docked, coverage,
distance to the station when hungry vs sated, and behavior entropy, per life, across
repeated lives; ablations for reflexes, each memory, the battery link, and all memory,
plus a no-engine random-walk baseline and a hand-wired upper bound. Headline: learned
reflexes lift mean lifespan from 1.26 h to 1.86 h (hand-wired bound 2.46 h); nothing
accumulates across lives yet.
The full program:
h_energy to see how much charging behavior is lost.Comparing ablated and full agents isolates which mechanisms contribute most to diverse, legible behavior.