The REFINE_TOOL statement uses vision to create or refine a tool offset. The REFINE_TOOL statement moves the robot gripper in front of an upward facing camera, takes a picture, and calculates a vision reference frame for the gripped part. The difference between the part reference frame center and the true gripper center is taken into account when the part is placed. This statement improves part placement accuracy for parts that are not gripped perfectly.
REFINE_TOOL {APPROACH --path--} REFINE --vision--
{AND --vision--} {WAIT --yes/no--} {SAVE_TOOL --tool--}
{{ALONG --path--} MOVE_TO --location-- {MOVE_ONCE --yes/no--} {DEPART --path--}{USING --tool--}} {OK_SIGNAL --o_variable--}
The sequence of operations performed by this statement is as follows (the relevant portion of the statement syntax is shown for each step):
Move to the picture-taking location1 specified in the picture record associated with the vision record. Perform the vision operation (REFINE --vision--).
If defined, rotate the gripper 180 degrees. This operation further compensates for inaccuracies in the camera-to-robot calibration ({AND --vision--}).
If specified, wait for all vision operations to complete before moving.
({WAIT --yes/no--}).
If WAIT is "no" and a MOVE_TO location is specified, enter the
{ALONG --path--} (if defined) and move to the approach segment of the MOVE_TO location (without the tool refinement).
Wait for the vision operation to complete. If it fails, exit with the optional OK_SIGNAL set to FALSE.
If specified, store the value in the tool record {SAVE_TOOL --tool--}. Set the current tool to the calculated value. If the vision operation fails, set the tool to undefined.
If a MOVE_TO location is specified, WAIT is no, and MOVE_ONCE is yes, do not move to the approach location with the refined tool since the move has already been made with the unrefined tool. Otherwise, if the ALONG path was not used, follow it now and then move to the approach location with the refined tool.
If defined, enter the optional depart path {DEPART --path--}.
If defined, set the output variable {OK_SIGNAL --o_variable--} to TRUE.
The requirements for the vision records used by this statement are described in the next section.
The REFINE --vision-- and AND --vision-- clauses of the REFINE_TOOL statement require a vision record that returns a vision frame defining the actual location of a part in the robot gripper. The center and orientation of this frame are used to calculate the true part location relative to the center of the gripper. If both vision clauses are used, the results of both operations are averaged to calculate the true part location.
The offset of the true part location from the actual gripper center will be used to compute a new tool transformation. The most commonly used vision tools that return a frame result are:2 blob finders, object finders, and computed frames. See the VisionWare Reference Guide for details.
The camera record for the refinement operation should have the following characteristics:
The camera used to generate the frame should be a stationary camera that the robot presents the part to.
The camera should have been calibrated with one of the fixed-mount calibration options (See Fixed-Mount Calibration Option)
The picture record for the refinement operations should have the following characteristics:
In the Robot Motion Information window, select Object moved by robot.
If Vision target location is selected, specify a path name in the Location data box. The path record controls the motion parameters for moving to the picture-taking location. These include, speed, motion type, etc. The robot-to-camera transformation created during camera calibration will be used as the basic picture-taking location. The location values in the path record are interpreted as an offset from this location. (The path reference frame must be set to World.)
When the robot moves the gripped part in front of the camera, it uses the location values created during camera calibration to move the gripper in front of the camera. The values in the path segment offset the camera from this location.
If you are using a computed frame to calculate the true part center, and the tools needed to compute the frame will not fit on a single screen, you can use multiple picture records with different path segments to move the camera to the different locations. MotionWare will take care of the necessary calculations to generate the frame.
Entering values in the Placement in FOV data boxes will offset the picture-taking location by the specified amount. The default values (50/50) put the center of the gripper in the center of the field-of-view.
Click Here to record the current robot offsets from the camera location in the specified path segment.
If Robot location is selected:
The first vision refinement operation (REFINE --vision--) will use segment 1 of the specified path, and the second refinement argument (AND--vision--) will use segment 2 of the specified path. If you enter a value in the Path segment data box, that segment will be used during both refine operations.
If your robot is using a tool transformation, teach all locations with that tool transformation as the currently selected tool (use Setup -> Display/Change Tool to verify that the correct tool is being used). The cameras used for the
REFINE --vision-- and AND --vision-- clauses must also be calibrated with the tool transformation in place.
The values generated by the vision operations are added to any existing values of the current TOOL to create the final placement location. When debugging the REFINE --vision-- and AND --vision-- clauses, observe the robot TOOL transformation before and after the refinement. If it changes drastically, either the camera calibration is incorrect or parameters on the picture record or associated path are set up incorrectly.
The current values of TOOL can be seen by selecting:
"Vision on the fly" is a feature that allows the vision system to take a picture while the robot is moving the part through the camera's field-of-view. Once the picture is acquired, the robot proceeds to a tool refinement location, stops at that location while a tool refinement is performed, and then proceeds to the place (drop) location.
To use vision on the fly, the vision system must be equipped with an I/O or external trigger (for example, a beam sensor). Also, a camera that supports asynchronous frame reset should be used for greatest accuracy.
To set up vision on the fly for applications where the Pick location is on a straight-line path to the Refine Tool location:
Design a path for the robot that places the Pick location (P) on a straight-line path to the Refine Tool location (R). See Example Robot Path for Vision on the Fly (Pick Location on Straight-Line Path to Refine Tool Location) for details.
Create a Path record and set up the path through the Refine Tool location (R). The record should contain a Path Entry location (E), a Refine Tool location (R), and a Path Exit location (X). See Creating Paths for details on creating a Path record.
Place the camera/trigger anywhere on the straight-line path between the Pick location (P) and the Refine Tool location (R). If no vision on the fly is desired, place the camera/trigger at the Refine Tool location (R).
Create a Picture record and select either the Wait for I/O signal or Wait for external trigger option. See the VisionWare User's Guide for details on creating a Picture record.
a. If the Wait for external trigger option is used, select Use strobe - Asynchronous to latch the corrected picture location.
b. If the Wait for I/O signal option is used, select Use strobe - Asynchronous to latch the corrected picture location. This will provide more accurate results than using picture recognition (hardware trigger versus calibration transformation).
Select Motion - Use it on the Picture record. Then, click Setup to display the Robot Motion Information dialog box.
a. Select the Object moved by robot check box.
b. Enter the name of the path created in step 2 in the PATH name data box.
c. Enter a value of zero in the Settling time data box.
d. Enter the path segment number that corresponds to the Refine Tool location (R) from step 2 in the Path segment data box. For example, if your path looks like Example Robot Path for Vision on the Fly (Pick Location on Straight-Line Path to Refine Tool Location), you would enter the value 2 in the Path segment data box.
Create a MotionWare sequence that contains a REFINE_TOOL statement. When a REFINE_TOOL statement is used, the robot will stop in the path (after the picture is acquired) and perform a tool refinement. An example sequence is shown below.
MOVE TO PICK
REFINE_TOOL REFINE vision_frame
MOVE TO PLACE
See the
MotionWare Reference Guide for more details on using the REFINE_TOOL statement in a sequence.
To set up vision on the fly for applications where the Pick location is not on a straight-line path to the Refine Tool location:
Design a path for the robot where the Pick location (P) is not on a straight-line path to the Refine Tool location (R). See Example Robot Path for Vision on the Fly (Pick Location not on Straight-Line Path to Refine Tool Location) for details.
Create a simple Path record with an Entry location (E1) and an Exit location (X1). The Exit location (X1) must be on a straight-line path to the Refine Tool location (R).
Create a second Path record and set up the path through the Refine Tool location (R). The record should contain a Path Entry location (E2), a Refine Tool location (R), and a Path Exit location (X2). See Creating Paths for details on creating a Path record.
Place the camera/trigger anywhere on the straight-line path between the Path 1 Exit location (X1) and the Refine Tool location (R). If no vision on the fly is desired, place the camera/trigger at the Refine Tool location (R).
Create a Picture record and select either the Wait for I/O signal or Wait for external trigger option. See Picture Records for details on creating a Picture record.
a. If the Wait for external trigger option is used, select Use strobe - Asynchronous to latch the corrected picture location.
b. If the Wait for I/O signal option is used, select Use strobe - Asynchronous to latch the corrected picture location. This will provide more accurate results than using picture recognition (hardware trigger versus calibration transformation).
Select Motion - Use it on the Picture record and then click Setup to display the Robot Motion Information dialog box.
a. Select the Object moved by robot check box.
b. Enter the name of the path created in step 3 in the PATH name data box.
c. Enter a value of zero in the Settling time data box.
d. Enter the path segment number that corresponds to the Refine Tool location (R) from step 3 in the Path segment data box. For example, if your path looks like Example Robot Path for Vision on the Fly (Pick Location not on Straight-Line Path to Refine Tool Location), you would enter the value 2 in the Path segment data box.
Create a MotionWare sequence that contains a REFINE_TOOL statement with an APPROACH path. When a REFINE_TOOL statement is used, the robot will stop in the path (after the picture is acquired) and perform a tool refinement. An example sequence is shown below.
MOVE TO PICK
REFINE_TOOL APPROACH PATH1 REFINE vision_frame
MOVE TO PLACE
See REFINE_TOOL Statement for more details on creating a MotionWare sequence that uses the REFINE_TOOL statement.
NOTE: In addition to the REFINE_TOOL statement, the vision on the fly feature can be used with a SET_FRAME statement (see SET_FRAME topic for details). To get the robot location from a SET_FRAME operation, do not select Object moved by robot in the Robot Motion Information dialog box of the Picture record.
A vision frame may use more than one camera to create the frame. If this is the case, the robot will move to each required picture-taking location.
Vision frames are based on the camera coordinate system, not the robot base, and are not interchangeable with reference frames from the frame database.