ZURICH—UBS AG's investment bank is feeling the squeeze.

A year after UBS mounted an ambitious effort to restore its investment bank to past glory, the Swiss bank's efforts have been undercut on several fronts. Poor pay packages have sparked an exodus of talent just a year after UBS went on an aggressive hiring spree.

And Switzerland's tougher capital requirements and the strength of the franc are making it harder to compete with foreign rivals.

As a result, some expect the bank to scale back its ambitions for the division, perhaps as soon as this summer. Analysts are watching next week's first-quarter results for any hint of a change in course.

While the entire investment-banking industry is grappling with new regulations and tougher competition, UBS is arguably in a tighter spot. In 2008, the bank wrote off more than $50 billion in securities, prompting a bailout from the Swiss government. Staff at the investment bank fell by a third. Swiss regulators then leapfrogged ahead of other countries in demanding especially large capital cushions for UBS and Credit Suisse Group to protect the country's two megabanks.