Background:

I am a final year Mechanical Eng student in Cork IT (Ireland) I am doing a project for the Irish Road Bowling Assoc. Accuracy-training facility is in planning for construction. I am to design an automated scoring system. The score is mainly judged on accuracy (the highest point of accuracy is the middle on the lane). I am looking for guidance in research and advice from similar experiences.
Simplified it is a steel bowl,60mm-800gramms "rolling" on a tarmac lane at 31m/s

Problem:
TO sense the position of the bowl relative to the gutter(low accuracy requirement). I am considering/researching the following:\
-Analogue photoelectric sensor at the side of the lane

-An array Inductive Sensors across the width of the track, under the tarmac

-Machine vision, with points of contrast. to help isolate the image.

Q1-Any comments on limitations of these technologies and what issues I might run in to down the line?

Q2- If the cost "per sensing line" will be relatively low then there will be several lines per each of the 3 tracks. The information will be processed and fed back to the players (including a computation of the stopping distance). Should I use a PC or a dedicated processor? Any guidance (recommendations of texts) on this and DAQ would be greatly appreciated!

Thank you in advance!

 

>-Analogue photoelectric sensor at the side of the lane

Reflective optical could be a problem with round shiny objects, as most of the light will be reflected away. This would need a good deal of testing.

>-An array Inductive Sensors across the width of the track, under the tarmac

Could be more robust, but possibly more expensive to build and install. Also, with many inductive systems there is a minimum permitted distance between sensors to avoid mutual interference. This would pose a limit to resolution.

>-Machine vision, with points of contrast. to help isolate the image.

Probably the most promising, but it may need overhead mounting. Also, would you be looking for an off the shelf vision system, or are you comfortable with writing some code? If the latter, you may want to investigate using a web cam to capture the image and then process it on the PC.

If you can get Curt Wuollet's attention on this list, he would be the person to talk to about this.

 

"Also, with many inductive systems there is a minimum permitted distance between sensors to avoid mutual interference. This would pose a limit to resolution."

He doesn't have to mount them in a single line perpendicular to the direction of ball movement. Mount them in two staggered lines or in a zig zag line. Whatever gives him the number of sensors he needs while avoiding the interference distance.

 

The staggered line is interesting, I have not thought of it before. I was considering using using Unshielded inductive sensors that would give me the widest coverage per sensor.

"Each sensor needs to be mounted at least three times its own sensing range away from the other. The use of an alternative frequency head on one of the sensors will prevent adjacent sensors’ sensing fields from interacting" (Elsivier Sensor Technology Handbook)

thank you,
Anton

 

> Reflective optical could be a problem with round shiny objects, as most of the light will be reflected away. This would need a good deal of testing. <

-- The "cannon ball" Bowl is not reflective, would the 60mm diameter diffuse the light too much?

>> -Machine vision, with points of contrast. to help isolate the image. <<

> Probably the most promising, but it may need overhead mounting. Also, would you be looking for an off the shelf vision system, or are you comfortable with writing some code? If the latter, you may want to investigate using a web cam to capture the image and then process it on the PC. <

I'm looking to more or less design a system.

Also the project is for a county-based sport, so I don't see them spending a lot of money on high performance vision systems for each lane. My concern is the speed at which the camera must operate at to be able to work on the image. The travel of the bowl could be as high 31 m/s.

> If you can get Curt Wuollet's attention on this list, he would be the person to talk to about this. <

I am absolutely new to this forum, how do i get attention of someone in a dignified way?

 

If you decide to do side photo sensors, consider one sensor as a measurement trigger looking perpendicularly across the track and a horizontal array of parallel beams, say one every inch for 12 inches or something, looking across at an angle.

When the ball breaks the perpendicular trigger beam, look at which beams in the angle array are broken, which will give you lateral track position. You can get a bar sensor to use in lieu of the array of sensors that will tell you where the break is withing its geometry.

If you do the brute force array, make sure to buy apertures from the manufacturer, or design them into the sensor mounting to avoid crosstalk.

 

> -- The "cannon ball" Bowl is not reflective, would the 60mm diameter diffuse the light too much? <

I have seen too many unpredictable results from reflective optical systems to want to state an opinion on that without testing. How things look to your eye and how they look to an optical sensor can be very different. You have to test this very carefully.

> Also the project is for a county-based sport, so I don't see them spending a lot of money on high performance vision systems for each lane. My concern is the speed at which the camera must operate at to be able to work on the image. The travel of the bowl could be as high 31 m/s. <

That is going to depend on what distance you are measuring over. For example, at 31 m/s over 10 m, that's 323 msec. If you need two web cams (one at each end of the measuring box) the bowl will pass through each field of view in a fraction of that time. I'm not sure that you *can* do this with a web cam, including capturing the image under field lighting conditions. However, the hardware is cheap and all off the shelf, and you said you are looking for 2D positioning as well as speed.

At this point I'm just punting some ideas out, I don't have a definite opinion on the best approach. I can however point out some of the problems that you will run into. The inductive sensor idea isn't a bad one, and it might be what you end up using. However, you need to sketch the idea out and see whether you will get enough resolution across the lane to suit your application.

You need to draw out some dimensions (e.g. distance to measure over) and come up with some numbers for resolution and accuracy, and then turn these into more detailed specs for what the system has to be able to do. E.g. millimetres of resolution, milliseconds of resolution, distance between the sensor and the target, etc. That will probably take some of these ideas right off the table.

>> If you can get Curt Wuollet's attention on this list, he would be the person to talk to about this. <<

>I am absolutely new to this forum, how do i get attention of someone in a dignified way? <

Just hope that he notices this and takes an interest. He's the one however that tends to have the low cost ideas provided that you don't mind getting your soldering iron out.

 

Yours is far more polite already than some of the discourse that gets my attention :^) The speed is a bit high for commodity cameras but could possibly be done with a trigger sensor up the lane that then grabs a single frame at the right moment. Video would be a great way to go because it can sense a large area with good resolution with a single device. And if it can be done with a commodity board camera and frame grabber card, the cost would be far lower than an array of typical automation sensors. The triggering part would be the hardest part as precise timing would probably involve interrupts. While at the printing plant, I was doing something much like this to verify address information printed on cards that were zipping by on an inserter. I had a frame grabber card that had hardware for a trigger on-board and it did work. I used an optical sensor to sense the target upstream and then with a delay, it grabbed the frame at the right moment. In your case it would be simpler since blur, to a degree, wouldn't be an issue and you are only concerned about one axis. So, you could grab the frame and threshold it to reduce to binary depth then simply find the edges and thus the center by pixel counting. You could resolve to something like 1/640 of your lane width which should be quite adequate. And you could store the pictures to resolve any arguments :^). I didn't take any of the code with me when I left, but it's fairly simple stuff once you have the image in a buffer. Does this sound like something you could get into?

Regards
cww

 

I want to thank all highly skilled people here for helping an undergrad engineer, I'm honoured!

I talked to my industrial partner today to clarify my budget limits. He gave me "a thousand or two" for proof of concept and if they like it then they well spend tens of thousands per each lane to put in a reliable, long-lasting outdoor system (I'll only be paid by earning myself a good degree I'd say).

Also if the each "sensing line" turns out to be cost-effective then there will be several of them per each lane, they will be used for more accurate shot DATA feedback to the player, including stopping position extrapolation.

Steve Myres,
Pardon me if I didn't understand you correctly, I just have several questions on your suggestion
Why do you reckon I would need a trigger sensor across?-To activate the array?
What type of photos. were you thinking of for the array(retro-reflective?) and why should they be angled rather than looking straight down onto the lane?

I looked into this type of solution and realised that because the diameter of the beams is so small I would need over a hundred sensors in-line to cover the 5m fully....It mightn't be a bad solution though.

M Griffin,

I am considering Machine Vision as one of the most promising solutions. Yes, I am only measuring one axis and yes, I am anticipating the added complexity of outdoor lighting and its variation throughout the year.
I completely agree that I have to make a list of engineering characteristics out of my customer requirements and I will get to it...soon
I have been coping without them because The required resolution is so low that I am just looking for the cheapest way to do it. (and in the process jumping the gun in the systematic design procedure)

Curt Woullet,

Machine vision is definitely one of the strongest possible solutions in my view as well. Especially if I don't have to go near the expensive industrial-type systems. I am worried however that the fact that the application is high speed and is in an outdoor (unstable) Irish Environment might drive me towards those industrial vision solutions.

Before I ask you a series of dimwitted questions on machine vision could I ask you to recommend a good educational resource on this topic please?

Question on the trigger: Can the camera not self trigger by the motion in its FOW? Is the point of the trigger to save memory?
The speed will vary from shot to shot, so if I do put in a trigger mechanism with delay will I then be able to tell the camera to take several frames (maybe: record until the OUT trigger has been crossed?) and then pick the best frame for processing?

I have not yet spent time on researching DAQ and processing as I am trying to pick the right sensor technology first. Would a good way to go be interfacing with LAbview and writing the code on it? I heard that you can buy a cheaper license for the read-only executable you design.

Thank you all for you guidance!

 

> Steve Myres, <

> Pardon me if I didn't understand you correctly, I just have several questions on your suggestion Why do you reckon I would need a trigger sensor across?-To activate the array? <

> What type of photos. were you thinking of for the array(retro-reflective?) and why should they be angled rather than looking straight down onto the lane? <

Yes the trigger tells you when to look at the array.

I was thinking see through photos, and not looking down on the lane but horizontally across it. The drawing at this link is a plan view (looking from the top).

http://members.cox.net/stevemyres/pics/Ball Detect.bmp

 

Very clever! I was locked into the thought that discrete sensors would have to be in transverse in a series. I now have to find out how expensive through beams are.

Thank you!!!!!!

 

> Machine vision is definitely one of the strongest possible solutions in my view as well. Especially if I don't have to go near the expensive industrial-type systems. I am worried however that the fact that the application is high speed and is in an outdoor (unstable) Irish Environment might drive me towards those industrial vision solutions. <

Environment is reasonably easy to deal with in this case since the cameras automatically adjust exposure and you can make the target a color that contrasts well with the bowl. And you would be shooting straight down which makes it fairly easy to deal with rain, etc. Any reasonably diffuse lighting would make night fun doable, just stay away from fluorescent or mercury or sodium lights as these pulse with the same line frequency the camera is synched to.

> Before I ask you a series of dimwitted questions on machine vision could I ask you to recommend a good educational resource on this topic please? <

You can ask, but I am afraid that I have not seen a good comprehensive source of information. Everything I did was on Linux so I started with the V4L project (Video for Linux) and grabbed tools that did what I needed, and either used them or learned from them to write software to do what I needed. Getting frames into a buffer is pretty much boiler plate as you might expect. From then on I pretty much wrote C code to do what I wanted. Thresholding is just reading through the array and assigning a bit in a matching array to a one or zero depending on whether the 8 bit value for that pixel is above or below a threshold, say, the average of all the pixels. The pixel counting is then reading line by line or column by column until you find zeroes starting from the edges. These are very simple to write and quite fast. No OTS software at the time did what I wanted to do, but it was all pretty straightforward. And since I had no need to display what was going on it was simply math and very small and fast. Then you can plot your measured lines on the original frame for the record. It's just reading and writing in arrays.

> Question on the trigger: Can the camera not self trigger by the motion in its FOW? Is the point of the trigger to save memory? The speed will vary from shot to shot, so if I do put in a trigger mechanism with delay will I then be able to tell the camera to take several frames (maybe: record until the OUT trigger has been crossed?) and then pick the best frame for processing? <

Aye, there's the rub. The camera can't, using commodity camera speeds. At 50 or 60 FPS (PAL or NSTC) you could miss the whole event. So the trick is to trigger in hardware early enough so that the frame that is captured is when the object is in view. This works because that's the only frame that is of interest. So rather than video per se, you have a still picture in effect. But you have all the information you need in that frame, in reasonably high resolution. 640x480 is common for NTSC, so you can get the equivalent of 320 sensors across the width measured. (One bit uncertainty). The triggered card was from Detect O Ray or some strange named outfit that I think did flame sensors originally.

> I have not yet spent time on researching DAQ and processing as I am trying to pick the right sensor technology first. Would a good way to go be interfacing with LAbview and writing the code on it? I heard that you can buy a cheaper license for the read-only executable you design. <

> Thank you all for you guidance! <

There is not much point in learning all this for one use or two. I might as well share.

Regards
cww

 

The problem with using Labview is that it's meant for a certain type of problem. If you can make your application fit the way that Labview works, then there are some advantages to it. If you aren't sure what you need to do, then you could end up stuck 90% of the way through without any good way of solving that lost 10% of the problem. If you really know Labview inside and out, and you understand the application problem thoroughly, then you can make that judgment. If not, then I would stick to conventional programming languages.

I would suggest prototyping the problem with as small and simple a program as possible to see if you can make the application even work. That will probably mean writing a simple command line program. Use that to get the hardware interfacing sorted out and to collect enough data to prove that the concept works.

When you prove that the idea works, you can then write a more user friendly program. For that, I would typically prefer Python and/or C, but you might make a different choice.

If this program is supposed to be widely used, then there are a couple of problems with using Labview. One is run-time license costs. Make an estimate of how many systems might be deployed over the lifetime of the software. If it's more than just a few, then look for something that has zero run-time license cost.

The other program is long term supportability. C is going to be around for as far in the future as is worth imagining. Python probably will be as well. Labview however depends on the viability and business strategy of NI. In addition, future versions may change in ways which are not compatible with current systems. If you are building a one-off system then you might decide the risk is not worth worrying about. If you have dozens of installations going to users who have never even heard of Labview, then that becomes a major concern.

 

Hi guys, sorry for the stalemate, has been pretty busy with exams and such. BTW you are all included in my referencing and bibliography in my mid-year report, thank you for the information.

I am going to check out Labview vision builder, it is supposed to have a large library.

 

I posted a question along with my previous reply, I don't know what happened to it but here it goes again:

-The bowl location in the FOV will vary with every shot. Is an algorithm to search and isolate the target complex?

Say I use a 60 FPS camera (which runs continuously?). this means that the exposure lasts anything from "0" to 0.0166 s right? Then the bowl travelling at 31 m/s would travel over 0.5 m during the maximum exposure correct?

1-Can I do anything with that column of blur? Strobe lighting is not useful unless you can synch it to the exposure time on the camera correct?

2- Could I use an asynchronous camera to eliminate the variation in exposure time and then just strobe for the duration of the full exposure?

Does it look like doing it doing this with a commercial camera is impossible? I am going into high FPS Asynchronous resetting etc..

Does a frame grabber support one camera at a time?
or would it be possible to connect and process 3+ cameras per card with external sensor triggering and strobe triggering for each one?

It feels like there is a steep learning curve ahead of me... :s

Thank you!

 

There is a method which was used with film photography but I don't know if it works with digital cameras (perhaps someone else could answer this). Instead of taking a "video", you would take a single image with two exposures. For high speed work, you would use a strobe, rather than relying on the shutter. You would then end up with a single image with the object (bowl) in two different locations.

You would need a simple but fast sensor (detector) of some sort to start the process. When the sensor detected the bowl, it would trigger the camera to open its aperture (or whatever the digital equivalent is). The strobe would then fire twice with a precisely timed interval between strobes (this probably needs to be electronically timed). This would give you a single image with two bowls. The relative position of each bowl would give you the vector, and the distance between them would give you the velocity. The timing of the strobe is what is critical here, not the timing of the camera. You just have to make sure the camera has a long enough exposure that both strobe events are captured. You also might need several (synchronized) strobes to avoid shadows.

To make this practical, you need to use a filter on the lens to filter out most of the ambient light, and a strong strobe. It would probably also help if the output of the strobe was chromatically matched to the filter (and the camera sensor) so that you got the best image response with the least ambient light. The mono-chromatic filter would also help with the image quality, as it reduces the effect of chromatic aberration in the lens. The bowl should reflect the chosen spectra as well as possible, and the background should absorb it as well as possible to give you the best contrast.

As I said however, vision systems are not my field of expertise, so I can't really promise that this would work in your particular application. Even an expert would need to get a camera, some filters, and a strobe and try it out in realistic conditions. The advantage to this approach however is that you just need a single image, not a video. Also as I said above, I'm not sure whether this concept works with digital cameras.

As for using Labview, unless you are really familiar with it I'm not sure I would recommend it. The problem tends to be that it will either do what you want, or it won't. If if doesn't do what you want out of the box, then the effort to work around the limitations tends to be a lot greater than just using a conventional programming language to begin with. Also keep in mind that there is more software in the end product than just the image processing bit. You are going to need a user interface to show what the thing is doing, and to allow the users to make any adjustments.

If I had to pick a recommendation out of the air I would suggest looking at using Python and the Python Imagining Library (PIL). Both are free, so you have nothing to lose by investigating them. PIL will convert image formats and apply thresh-hold values and filtering.

As Curt Wuollet said, the first thing you want to do is to convert the image into a 2D array of boolean on/off values. Once you have that then you start to apply basic mathematics to do things like calculate the centroid of the image (Google will find lots of explanations of that). You may have to apply several algorithms successively to narrow down the locations. You have the advantage that you know the approximate size of the bowls so you can apply some short cuts to your logic. At this point however, it's a mathematical problem. As an additional hint, I would suggest that once you think you have found the bowls in the image, draw a cross hair on the centroid of each and display that. That will help a lot in debugging both initially, and in setting the system up in the field.

If you want to see if this is practical, then you need to take a camera a collection of filters, and strobe and try getting some sample single images under field conditions (although that might be a bit difficult at this time of year). You can create a mock-up of the double image by combining two shots with image editing software and then try applying your algorithms to that to see if you can derive results from the data.

This isn't really my field, so take the above with a grain of salt. The double-image idea may not work with modern digital cameras, so that is the first thing you would want to investigate. Perhaps Curt Wuollet could comment.

 

So why not have an edge detector photo and use that to trigger the strobe? I've done that before, though not with as fast-moving a target as this, but a strobe triggers almost instantaneously, so I don't think it should be a problem. You might have to lead the target zone with the photo by an inch or two, and the longitudinal position of the bowl will still vary with velocity, but should still be within the frame.

 

> I posted a question along with my previous reply, I don't know what happened to it but here it goes again:

> -The bowl location in the FOV will vary with every shot. Is an algorithm to search and isolate the target complex?

> Say I use a 60 FPS camera (which runs continuously?). this means that the exposure lasts anything from "0" to 0.0166 s right? Then the bowl travelling at 31 m/s would travel over 0.5 m during the maximum exposure correct? <

Running continuously is probably not a good mode as you need to synchronize with the target and not the power line. NTSC and I think, PAL will try to sync with the line for reasons beyond the scope of this discussion but if you run the camera on DC and trigger a single frame you have your best chance of catching the event.

> 1-Can I do anything with that column of blur? Strobe lighting is not useful unless you can synch it to the exposure time on the camera correct? <

Yes, you don't care about the blur in the direction of motion. Whether you get a
sharp circular image or a column of blur, given a symmetrical circle of
confusion, the edges and center should be reasonably accurate. Strobe lighting
would be costly and not necessary as the blur doesn't affect the information you
need to extract.

> 2- Could I use an asynchronous camera to eliminate the variation in exposure time and then just strobe for the duration of the full exposure? <

Yes, you could certainly get equipment that would "stop" the motion, but the cost would increase exponentially.

> Does it look like doing it doing this with a commercial camera is impossible? I am going into high FPS Asynchronous resetting etc. <

Challenging perhaps, but not impossible. You do have to get clever and focus carefully on what you need. If you want nice images of the target, that will be expensive and difficult. If you want to score based on the position, that is much easier and I think there is a good chance you can do that with commodity cameras and cards of the type intended for the security market. The cost difference is quite striking. If you use commercial software, you are limited to what cards, cameras and algorithms they support and it may well be impossible or cost prohibitive to do what _you_ want to do. That's why I was writing definite purpose software. None of the available applications would do what I needed to do, even though it was quite straightforward. That's the problem with shrinkwrap software, if there's no button for it, you can't do it :^).

> Does a frame grabber support one camera at a time? <
> or would it be possible to connect and process 3+ cameras per card with external sensor triggering and strobe triggering for each one? <

The card I used for the triggered capture had four inputs, but I was only using one. I found some stuff for a PXC200 Imagenation card, but I can't be sure that I was using that. I left all the code there. The cameras came from Polarisusa.com They were fairly cheap at the time. Driver was by Rubini.

> It feels like there is a steep learning curve ahead of me... :s

I found the learning quite rewarding, but that's one of my idiosyncrasies.

Regards
cww

 

>If you decide to do side photo sensors, consider one sensor as a measurement trigger looking perpendicularly across the track and a horizontal array of parallel beams, say one every inch for 12 inches or something, looking across at an angle. <

---- snip ----

I was looking into cheaper photo sensors but I am afraid that the receivers might be triggered by sunshine. Using Laser photosenors means that the price will be huge. Do you think through-beam or retroreflective be most appropriate for the array?

Also the main thing I want to know is how would I monitor this array of 100+ discrete sensors? DAQ devices have low count of possible inputs and I hardly need all their features. Is there a way that I can, for example, have a node by the array which will take a command trigger, read the array, and then send a serial code of the state of the array to the pc over long distance? over ethernet..
Thank you all again.
The more I'm learning about what I need to do to solve this application the more I realize that I know so little....makes you wonder what I have learned in the last four years
Anton

 

"I was looking into cheaper photo sensors but I am afraid that the receivers might be triggered by sunshine. Using Laser photo sensors means that the price will be huge. Do you think through-beam or retroreflective be most appropriate for the array?"

I don't think most normal photos will be spoofed by sunshine. In a lot of cases the emitters are modulated so the receiver knows it's looking at an emitter. (As I said before, make sure to design the receiver mounting to be directional enough so you're sure it's seeing the right emitter) If the sunshine caused any problem at all it might be overwhelming them and keeping them from triggering, but just ask the distributor and make sure the sensor you're buying will work in sunlight. I like through-beams but retros could work too. Check the range.

"Also the main thing I want to know is how would I monitor this array of 100+ discrete sensors? DAQ devices have low count of possible inputs and I hardly need all their features. Is there a way that I can, for example, have a node by the array which will take a command trigger, read the array, and then send a serial code of the state of the array to the pc over long distance? over ethernet.."

I wasn't thinking anything like 100 sensors. My drawing shows one trigger and eight skew. You'd probably want more than eight, but somewhere in that order of magnitude. You should get enough resolution with WAY less than 100 sensors, unless I don't understand your goals. Yes, you could do this with a PC, especially with less than 20 sensors, but why? I can see it if you're doing the vision, but if you're going the sensor route, buy a PLC! You can get them for $100-200 that will take that number of sensors, come equipped with an interrupt input for the trigger, and can be addressed with Modbus serial or TCP if you need to display and/or log the results on a PC.