Sequential images, which composed of a stack of 2D static images in order arranged one by one, contain not only information that static images carry, but also information of motions hidden among frames, the latter appearing to be more important. To detect this information, optical flow detecting technique, which is a well-known method that applies computer vision theory to detection of dynamic information, analyzes the velocity field of each pixel in images from the viewpoint of statistics [1, 2]. According to our experience working on contour-based optical flow detecting [3], it always suffers from many difficulties on mathematic analysis when it is used to solve the problem on deformable objects and non-continuous view angles. Furthermore, there exist many statistic, conjectural and uncertain problems as far as specific understanding of velocity is concerned. Therefore many new detecting methods on Echocardiography Sequences [4-8] have been developed. In this study, we have searched for a new method to detect dynamic information from sequential images, which can rebuild gray (position) ~time function (waveform) on direction lines to detect dynamic information on moving views, moving objects, and moving characteristic points in the images.

Rebuilding a gray (position)~time function (waveform) on an arbitrary direction line is to reposition the recognized gray level of views, objects or characteristic points on that direction line and further to confirm its position by capturing the white dots on the line. Because these direction lines are artificial, the positions of those white dots can be confirmed. On the other hand, views and objects in the same frame of sequential images are relatively still, leading the white dots can be confirmed uniquely, while altering with the shifting frames of sequential images. Accordingly, with the certainty of the time interval between two neighboring frames, these white dots, which move originally on the direction lines, are extended with the time. Consequently the gray (position) ~ time waveform (function) on direction lines are obtained. Moving equation on direction lines are obtained when these dots are confirmed as moving objects. From the point of kinematics, the most important moving basic information data can be obtained. Therefore we can mine data in depth and find correlative new dynamic information with the combination of kinematics and dynamics.

The movements of viewing angles, objects or characteristic points of practical images, however, are very complex. Therefore it is impossible that they simply move back and forth on a single direction line in most cases. As a result, the setting of direction lines of sequential images in a practical application should be a combination of many approaches so as to detect practical information and to engage the tracked target points with the combination of the settings of direction lines and certain algorithms. In this paper, two methods are developed to solve this problem. One is to catch the bright points in the ECG a form of single directional stepping translation with frames. Another is to obtain a non-functional motion model that is in turn applied on the direction line to capture functional moving points. Both methods are employed in practice [9, 10]. According to the moving situations of image objects, more direction lines can be set, leading to more data and new accurate dynamic information being mined and detected.

In this paper, a method has been developed, which combines with certain algorithms, which focuses on the tracking of moving objects to set the direction lines, and is based on the movement of moving objects with the help of sequential images, composed of static images.

As discussed above, direction lines are used to track moving objects, so the optimal direction line is just the trace of a moving object being tracked. However, finding a direction line from images is difficult if not possible. To this end, what we can do is to set artificially some controllable and positionable direction lines which can capture the predefined view, objects and characteristic points. And then we can combine with some related algorithms to set direction lines to obtain a moving trace of the engaged objects. That is gray (position) ~time function, which first-order differential is the velocity of movement, and second-order is its acceleration. Thus, we can mine various moving information.

The method used to set direction lines can be classified into two kinds. The first is the setting of fixed direction lines and the second is the setting of direction line variables within frames. The details are given below.

2.2. Setting of Direction Line Variables with Frames

With this setting, the position coordinates of a direction line are keeping altering and changing with the altering of sequential frames. It can be an arbitrary number on any arbitrary direction lines, as shown above. Therefore, this method is more complex than the setting of fixed direction lines. However, it may be a way to track more complex moving objects. As mensioned above, the major idea of the method is that all of the direction lines should be controllable and positionable, which requests the altering of this kind of direction lines should be regular and operated according to a certain algorithm so that they can be positioned in different frames of a set of sequential images. The detailed setting of altering direction lines can be divided into the following two subclasses:

• Single directional stepping translation form with frames
• Complex moving form with frames

The direction line of single directional stepping translation form keeps altering with frames. In this way, we can track the moving trace of an object, which moves very complexly within the range of R (the longest deviate distant to the center- line), as illustrated in Fig. (1). Where the length of the direction line is 2R, and the line is perpendicular to the centerline. The stepping translation of the direction line between two neighboring frames is d, and there is a direction line corresponding to each frame to track an object point. As a result, the moving trace of the object can be extracted from the gray (position) ~ time function (waveform), similar to sampling techniques.

Fig. (1). The setting of direction lines varied with frames in the tracking of moving objects.

With this direction line in practice, the moving trace of the ECG object on the lower part of echocardiography can be obtained, illustrated in Fig. (2). We can join it synchronized to Omnidirectional M-mode Echocardiography (which will be introduced in the subsequent introduction) as their time reference.

Fig. (2). The application of setting of direction lines variable with frames in the tracking of the bright spot object of ECG on echocardiography.

2.3. Nonfunctional Motion and Functional Motion

Normally, it is very complex to set a direction line of moving form within frames, but it may track more complicated moving traces, leading to a separation of number of important movements from the other movements, which is the main intension of this study. Using this method, we will evaluate Omnidirectional M-Mode Echocardiography functional motion separation, the important issue when it comes to detect every structural function of cardiac to a great extent of precision. As the equipment that is used to non-traumatically detect cardiac moving image is given a fixed position on an external body, the detected sequential motion images of one cardiac structure are only a general result, which consists of functional motion of this self structure (such as diastole and systole) and other nonfunctional motion (such as blood flow, breathe, the dragging between each structure.). Hence it can’t single out the functional motion that can reflect the inherent information of one structure, the most obvious disadvantage of these medical dynamic images as illustrated in Fig. (3a). If the first and the last coordinate of the fixed coordinate line(x, y) is (105, 30; 136, 112), the cardiac structure of being captured from No.1, 14, 34 frame in the sequential images is in three different position, they are aortic front wall, ventricular septum and aortic front wall. On the other hand, under these three pictures, the corresponding rebuilding echocardiography is illustrated. Thanks to the influence of non-functional motion, the setting of fixed direction lines can’t track point A of this structure nor capture the functional trace of this point at all times. In the current international standard, it is the momentous unfathomed problem to detect dynamic image information.

Fig. (3a). Fixed direction lines capturing gray point and rebuilding M-mode cardiogram in sequential images (No.1frame, No.14frame, No.34frame).

Fig. (3b). Complex moving direction lines by the action of non-functional motion to capture gray point and rebuilding M-mode cardiogram in sequential images.

By statistic moving of nonfunctional motion character points which are close to point A, we obtain the nonfunctional motion of these points in the sequential images of Fig. (3a), and act it on the direction line according to the principle of relative motion, consequently make the direction line move complicated. From Fig. (3b) we can seethe first and the last coordinate of direction line in No.1,14,34 frame is (105, 30; 136, 112) , (125, 28; 156, 110), (93, 37; 124, 119). On the whole these points are all the positions of A of this structure, thus the rebuilding M-mode echocardiography is clearer, and synthesize the three lines near No.14 frame into one line.

Omnidirectional gray ~ time echocardiography waveform, or called Omnidirectional M-mode Echocardiography, is the most typical example of the setting of fixed arbitrary number direction line, and is a fruitful method in our research. According to the rebuilding principle of Omnidirectional gray ~ time waveform, if the direction of a sample line selected is the same as the actual motion direction, the gray intensity data of the pixels on them (e.g. B) can generate gray ~ time waveform combined with corresponding temporal relationships. As shown in Fig. (4), the gray ~ time waveform of three lines is rebuilt from long and short axis echocardiography respectively and represents motion waveform of the wall of ventricle structure, which is because these gray values can stand for the boundary of cardiac structure in ultrasound images. Furthermore, all these resulted M-mode echocardiography are synchronised in time, which reflects the movements of different parts in different cardiac structures.

Fig. (4). The setting of three fixed direction lines rebuilding gray ~ time echocardiography wave- form of motion orientation in long axis (the right) and short axis (the left) of cardiac structure.

As for Omnidirectional gray ~ time waveform detected from echocardiography sequential images, we can select arbitrary direction of any parts in any structures to set sample line (direction line), which can be used by to follow the motion tracks of this part. Thus, sample lines are not confined by the direction of ultrasound wave-line any longer. Consequently a substantial improvement in detection accuracy is obtained. Moreover, we can compare the motion parameter of every echocardiography and their temporal-phase relationships, because any arbitrary number sample line whose waveform is synchronous in time can be selected. As a result, the idea of Omnidirectional M-mode Echocardiography Waveform System has been patented by the National Invent Patent of China (No. ZL 98 125713.5).

3.1. LEJ-1 Omnidirectional M-Mode Echocardiography System

Subsequently, we manufactured LEJ-1 Omnidirectional M-mode echocardiography system with the setting of fixed direction lines. The system facilitates two functions, namely automatic operation and manual operation. In the former, when confirmed a center, 12 oriented direction lines will give out every 30 angles, which looks like a clock panel. Then their gray ~ time waveform of specified duration will expend in screen or be printed on laser printer paper, as given in Fig. (5). This kind of operation is mainly suited for left ventricle short axis sequence images whose motion is mainly acentric and centripetal.

Fig. (5). Omnidirectional gray ~time echocardiography waveform in automatic operation.

The second operation is manual operation. In accordance with any positions, any structures in echocardiography images, based on their motion direction, operator can select any direction sample line with mouse and set manually to build their Omnidirectional echocardiography gray ~ time waveform lasting for 5 seconds. In this operation, there will be a preview (echocardiography waveform) displayed on the monitor to give a choice on whether a sample line is selected.

Regardless which operation is carried out, a series of resulted M-mode echocardiography waveforms corresponding to a sampling line can be chosen to perform further detection for various parameters. Parameters such as amplitude, velocity, compressing peak velocity and the quality of thickness, which related to the motion of cardiac structure, are detected from appointed sample line in echocardiography waveform as demonstrated in Fig. (6). According to the description above, we rebuild ECG below echocardiography which is synchronized with all of Omnidirectional M-mode echocardiography waveform so as to compare Omnidirectional gray ~ time waveform with ECG (see Fig. 2).

Fig. (6). A lot of Omnidirectional of gray ~ time echocardiography Waveform in manual operation Comparative waveform with ECG.

The most extinctive difference between our Omnidirectional gray ~ time waveform system and Anatomy M-mode Echocardiography [19] is that we have obtained synchronous Omnidirectional gray ~ time waveform group on any parts, any direction and arbitrary columns (corresponding to arbitrary sample lines) based on dynamic information detected from sequential images with synchronous ECG, whereas. The anatomical M-mode echocardiography shows only two or three images simultaneously even when they come from a different cardiac period.

Compare with Tissue Doppler echocardiography [20], our system has some characteristics as specified in the follows [21]:

• the accelerator comes from differential coefficient to athletic function directly not from frequency excursion indirectly.
• the orientation can changes freely.
• Groups of arbitrary direction lines in any part of any structure can be set and presented corresponding to Omnidirectional echocardiography images with synchronous ECG so as to be compatible with waveforms and temporal phases.

The dynamic analysis here we made in sequential images is similar to amplifier analysis made for several object units in static images. One example is that an image of streets in a static whole scene may only contain lines. However, when further amplified, these lines can display cars, shops, or other details in these streets. Under the similar pattern, the sequential images show the “whole scene” of every cardiac structure from its full view. It can be implemented by the way of Omnidirectional gray ~ time wave- form detecting method to acquire detailed knowledge of the motion status, namely dynamic details of any parts in any structures (including motion waveform, amplitude, velocity, and accelerate, etc. when the point’s movement being along with the time). The wave- form, which we obtained, is the motion trace or motion equation of the detecting part in cardiac structure along with time. On the base of it, one-order timing of differential of Omnidirectional gray ~ time wave- form is the velocity of detecting movement of cardiac structure and two-order is acceleration . As follows Fig. (7).

Fig. (7). Omnidirectional M-mode echocardiography images with its velocity and acceleration functions.

In summary, it is the most important to obtain new dynamic information in the hemodynamics study. So we implement the data mining of new dynamic information from every cardiac structure from sequential images. We have applied the method to the analysis of cardiac B- scan/color scan (or echocardiography) sequential images. In fact, this motion analysis method on sequential images can also be employed to many other medical sequential images such as X-ray, coronary-contrast, etc. Consequently, this method for motion analysis can also be improved and developed further to fit the characteristics of each imaging modality.

With the advanced development of science and technology, the method detailed in the paper and used to analyze dynamic information is becoming much profounder. The main methods currently used can be classified as a viewpoint-based approach, which is employed on sequential images, focusing on dynamic objects and on movement. While based on sequential images, the method proposed in this paper constitutes all its syntaxes, such as the setting of the direction lines and the algorithm combined, for moving objects principally. So it has some characteristics such as understanding motion details of each cardiac structure, confirming moving information data, and correcting detecting results. To this end, the prior knowledge bases are required in order to set direction lines easier to analyze the motion information.

At present, it is required that all pixels are relatively stationary in a single sequential image. The time interval between two neighboring frames is 1/25 second in the PAL system and1/30 second in the NTSC system, which is a very long time for usual objects. So, if the time of frame interval is reduced farther, the dynamic information detection of higher-speed movement can be obtained with this proposed method.

In addition, the direction lines used above are all straight lines. In the future, if they are modified into controllable, positionable, and non-linear direction lines, it will be easier to track more complex movement leading to the improvement of this approach further.