Assuming you've lost the preload adjuster screw (which I think is what you describe). I'd take the fork cap off and put a bolt right up through the hole, put a washer on each side of the cap and a nut and clamp the washers together. This should I reckon should securely close off the hole?

__________________