You have to know the name or the object associated to all the windows you have to close.

To close one window that you opened with:
var win1 = window.open(...);
you have to give the JS command:
win1.close();

If you don't know the object of the window, or you have lost it (e.g while you have changed the page in the main window), you have to re-open the window:
var win1 = window.open('','NAME_OF_WINDOW');
win1.close();

You can also put the code inside a loop, but the code in this case depends too much on the context, so it is difficult to explain without knowing more details of your specific problem.