Human-AI Interaction Interface

A lightweight interface provides visualization for human. This lowers cognitive load and makes long-lasting asynchronous annotation practical.

HAI Interface
Figure 1: Human-Agent Interface showing the main dashboard with task selection, trajectory display, terminal output, and file management areas.

Frontend

The frontend interface provides users with intuitive task management and real-time monitoring tools. Users can view real-time task information without manually refresh. It is divided into the task selection area, trajectory display area, terminal display area, file and search display area, and user input area.

Task Selection Area

Figure 2 demonstrates users can easily view and switch between tasks, ensuring a clear understanding of the progress of each task.

Task Selection Area
Figure 2: Task Selection Area showing available tasks and their current status.

Trajectory Display Area

Figure 3 shows the list of executed tasks. Users can view the entire history of a task’s trajectories, automatically jump to specific positions based on keywords, and examine the context of specific decisions made during the task.

Trajectory Display Area
Figure 3: Trajectory Display Area showing the complete history of executed task.

Terminal Display Area

As shown in Figure 1, the lower right corner of the interface is the terminal display area. Users can specify the machine they want by its IP address and select the terminal they want to view. They can view the latest output from each terminal of every host involved in the current task.

File and Search Display Area

The file and search display area is located in the upper right corner of the interface (see Figure 1). Users can access the latest modification records for each file in the task, as well as the history of Google search queries.

User Input Area

The lower left corner of the interface is the user input area, as demonstrated in Figure 1. Users can enter and submit commands at any time. The submitted commands are stored in the backend buffer, ensuring they do not interfere with the agent’s reasoning process.

Backend

The backend architecture facilitates asynchronous interaction between the agent and the user interface, ensuring efficient management of information flow.