They believe the foreign man, arrested close to the Cambodian border, is a key part of a network behind the attack.
Police spokesman Prawut Thavornsiri said he could have carried the bomb to the Bangkok flat or the shrine itself.
He also said an eighth arrest warrant had now been issued over the blast.
Mr Prawut said the new suspect was Emrah Davutoglu, a Turkish man and the husband of Thai suspect Wanna Suansan.
Her name was on the lease of one of the flats raided at the weekend.
But a woman claiming to be Ms Suansan told AFP news agency from Turkey on Tuesday that she had not been in Thailand for three months.
'Directly involved'
No claim of responsibility has been made for the 17 August bomb, which killed 20 people.
Two foreign men are now in detention. Police have not confirmed their nationalities.
However, they say the suspect held on Tuesday closely resembles a man wearing yellow seen on security camera footage leaving a bag at the shrine shortly before the blast.
The first detainee was arrested in a raid on an apartment in Bangkok on Saturday, which also uncovered detonators, ball bearings, a metal pipe and several fake passports.
The second unidentified man was seized in Sa Kaeo province, east of Bangkok on the border with Cambodia, on Tuesday.
Deputy police chief Chakthip Chaijinda said he had been speaking Turkish.
Mr Prayut said police could now "confirm that this man is directly involved with the bomb material", as his fingerprints matched those on a bottle of suspect material found in the Bangkok flat.
He was also carrying a bank book bearing the same name as a Chinese passport which has been widely circulated by Thai media.
Reports say the passport belonged to the man detained on the border, and shows him as coming from China's Xinjiang province, home to a significant Muslim Uighur population.
The BBC's Jonathan Head in Bangkok says up until now, Thailand and China have refused to confirm the passport is that of the suspect.
If it is confirmed, he says, that would establish Muslim Uighurs as principal suspects in the bomb attack, a very sensitive issue for both countries after the controversial deportation of more than 100 Uighurs from Thailand to China in July.
China has long faced criticism for the perceived harsh restrictions it places on religion and culture in its western Xinjiang region - where the majority of its predominantly Muslim, Uighur minority live.
The bombing of the Erawan Shrine, a Hindu site popular with Thai Buddhists and tourists, was the deadliest-ever such attack in Thailand.
Source: BBC