Modeled a virtual RGB pinhole camera on the end effector of a seven-joint KUKA iiwa arm in PyBullet, loaded from its URDF. Derived the 3D-to-2D measurement equations from the camera pose and field of view to predict where a target appears in the image.

Validated the model against independent OpenCV detections while varying camera translation, tilt, field of view, target size, and line of sight, with a baseline error of 2.91 pixels on a 640 × 480 image.

PyBullet simulation of a KUKA arm beside its end-effector camera view, with the red cube detected
PyBullet simulation and end-effector camera view with the OpenCV detection.
Gray obstacle blocking the camera's view of the red cube
Line-of-sight test with an obstacle blocking the target.