Japan's Finance Minister said Tuesday the country would not enter a bilateral free trade agreement with the U.S.