Today I added a detection system for the camera. I made so that the camera must be stationary to be detected as the piano. I just simply recognized the piano from the frame, compare the bounding box from the previous frame and check the difference of the bounding box. To figure out whether the bounding box has changed from the previous frame, we just calculate the overlapping area of the piano's bounding box from the current frame with the previous frame's bounding box.

bool PianoRecognition::process_piano_calibration(void) {
    ...
    std::vectorcontour;
    Rect bounding_rect;
    std::vectorprev_contour;
    Rect prev_bounding_rect;

    int overlap_streak = 0;
    const int overlap_streak_threshold = 50;
    while(1) {
        ...
        if(recognize_piano(frame , contour) == false) {
            putText(debug_copy , "No piano found!" , Point(0 , 30) , FONT_HERSHEY_SIMPLEX , 1 , Scalar(0xff , 0x00 , 0x00) , 3);
            // reset the streak
            overlap_streak = 0;
        }
        else {
            bounding_rect = boundingRect(contour);

            std::vector>contour_list = {contour};
            drawContours(debug_copy , contour_list , 0 , Scalar(0x00 , 0xff , 0x00) , 4);
            rectangle(debug_copy , bounding_rect , Scalar(0x00 , 0x00 , 0xff) , 4);
        }

        // calculating the area overlapping with the previous bounding rectangle
        double area_overlap = (bounding_rect & prev_bounding_rect).area();
        
        double area_current = bounding_rect.area();
        std::cout << "area_overlap = " << area_overlap << "\n";
        std::cout << "area_current = " << area_current << "\n";
        // Check whether the overlapping are is larger than 90% of the current area
        if(area_overlap >= area_current*0.90 && area_overlap != 0) {
            // Increase the streak
            overlap_streak += 1;
            std::cout << " ---- overlap_streak : " << overlap_streak << "\n";
            
            // the piano is finally recognized
            if(overlap_streak >= overlap_streak_threshold) {
                cv::RotatedRect rotated_rect;
                rotated_rect = minAreaRect(contour);
                
                set_detected_piano_img(frame , contour , rotated_rect);
                break;
            }
        }
        ...

        // copy prev
        std::copy(contour.begin() , contour.end() , std::back_inserter(prev_contour));
        memcpy(&prev_bounding_rect , &bounding_rect , sizeof(Rect));
    }
}

This code simply calculates the piano's bounding box of both previous frame and current frame. If the overlapping area between two bounding boxes are larger than 90% of current area, the camera is considered to be stationary and increases the streak. If the piano has disappeared from the sight, the streak goes back to zero. If the streak exceeds certain threshold, the system thinks that the camera has been sufficiently stationary and capture the last frame and uses for the analysis of keyboards.

I actually tested the system, and it worked really well! Now I really need to make the keyboard detecting system..!!