The 35-year-old left-hander had given Ireland hope of a first ODI victory over a Test-playing nation on home soil right until he was dismissed, stumped by Luke Ronchi off the bowling of left-arm spinner Mitchell Santner, who went on to complete his first five-wicket ODI haul.
New Zealand set Ireland a target of 290 to win the second match of the ODI tri-series in Dublin with Neil Broom top scoring for the Blackcaps with 79.
He finished with 5-51 and was named player of the match.
Ireland were 26-2 after debutant Kuggeleijn (3-41) accounted for openers Williams Porterfield (12) and Paul Stirling (14).
Ireland also gave a debut to off-spinning all-rounder Simi Singh, who came in for Stuart Thompson in the only change to the team that played Bangladesh in the opening game of the series on Friday.
New Zealand's next game is against Bangladesh on Wednesday.
Taylor and Worker steadied an early New Zealand wobble at 55-2 to add 84 runs for the third wicket, while Broom shared a 79-run union with Jimmy Neesham (30) as the Black Caps rode out their innings with few hiccups and posted a competitive 289-7.
For Ireland medium-pacers Tim Murtagh and Barry McCarthy each took two wickets.
Ross Taylor scored 52 and George Worker 50 after Ronchi (37) and captain Tom Latham (15) departed inside the first 10 overs. Worker, in only his third worldwide, was the more patient while Taylor scored at a run-a-ball.
Both reached their 50s, but both were out immediately afterwards, Taylor caught by the Irish captain at point and Worker, whose 50 took 88 balls, with just three fours, caught at deep mid-wicket.
It was Santner's third spell with the ball and a change of ends that made the difference as he took four wickets for seven runs in four overs to end Ireland's fearless chase of 290 on 238 in the 46th over.