CLNS 94/1287 CLEO 94-15 June 20, 1994

Search for neutrinoless decays of the tau lepton.

J. Bartelt,^{1} S.E. Csorna,^{1} Z. Egyed,^{1} V. Jain,^{1} K. Kinoshita,^{2} B. Barish,^{3} M. Chadha,^{3} S. Chan,^{3} D.F. Cowen,^{3} G. Eigen,^{3} J.S. Miller,^{3} C. O’Grady,^{3} J. Urheim,^{3} A.J. Weinstein,^{3} D. Acosta,^{4} M. Athanas,^{4} G. Masek,^{4} H.P. Paar,^{4} M. Sivertz,^{4} J. Gronberg,^{5} R. Kutschke,^{5}

S. Menary,^{5} R.J. Morrison,^{5} S. Nakanishi,^{5} H.N. Nelson,^{5} T.K. Nelson,^{5} C. Qiao,^{5} J.D. Richman,^{5} A. Ryd,^{5} H. Tajima,^{5} D. Sperka,^{5} M.S. Witherell,^{5} M. Procario,^{6}

R. Balest,^{7} K. Cho,^{7} M. Daoudi,^{7} W.T. Ford,^{7} D.R. Johnson,^{7} K. Lingel,^{7} M. Lohner,^{7}

P. Rankin,^{7} J.G. Smith,^{7} J.P. Alexander,^{8} C. Bebek,^{8} K. Berkelman,^{8} K. Bloom,^{8} T.E. Browder,^{8}^{∗} D.G. Cassel,^{8} H.A. Cho,^{8} D.M. Coffman,^{8} D.S. Crowcroft,^{8} P.S. Drell,^{8} R. Ehrlich,^{8} P. Gaidarev,^{8} R.S. Galik,^{8} M. Garcia-Sciveres,^{8} B. Geiser,^{8} B. Gittelman,^{8} S.W. Gray,^{8} D.L. Hartill,^{8} B.K. Heltsley,^{8} C.D. Jones,^{8} S.L. Jones,^{8} J. Kandaswamy,^{8} N. Katayama,^{8} P.C. Kim,^{8} D.L. Kreinick,^{8} G.S. Ludwig,^{8} J. Masui,^{8} J. Mevissen,^{8} N.B. Mistry,^{8} C.R. Ng,^{8} E. Nordberg,^{8} J.R. Patterson,^{8} D. Peterson,^{8} D. Riley,^{8} S. Salman,^{8} M. Sapper,^{8} F. Würthwein,^{8} P. Avery,^{9} A. Freyberger,^{9} J. Rodriguez,^{9}

R. Stephens,^{9} S. Yang,^{9} J. Yelton,^{9} D. Cinabro,^{10} S. Henderson,^{10} T. Liu,^{10} M. Saulnier,^{10} R. Wilson,^{10} H. Yamamoto,^{10} T. Bergfeld,^{11} B.I. Eisenstein,^{11} G. Gollin,^{11} B. Ong,^{11} M. Palmer,^{11} M. Selen,^{11} J. J. Thaler,^{11} K.W. Edwards,^{12} M. Ogg,^{12} A. Bellerive,^{13}

D.I. Britton,^{13} E.R.F. Hyatt,^{13} D.B. MacFarlane,^{13} P.M. Patel,^{13} B. Spaan,^{13} A.J. Sadoff,^{14} R. Ammar,^{15} S. Ball,^{15} P. Baringer,^{15} A. Bean,^{15} D. Besson,^{15} D. Coppage,^{15} N. Copty,^{15} R. Davis,^{15} N. Hancock,^{15} M. Kelly,^{15} S. Kotov,^{15} I. Kravchenko,^{15} N. Kwak,^{15} H. Lam,^{15} Y. Kubota,^{16} M. Lattery,^{16} M. Momayezi,^{16} J.K. Nelson,^{16} S. Patton,^{16} D. Perticone,^{16} R. Poling,^{16} V. Savinov,^{16} S. Schrenk,^{16} R. Wang,^{16} M.S. Alam,^{17} I.J. Kim,^{17} B. Nemati,^{17}

J.J. O’Neill,^{17} H. Severini,^{17} C.R. Sun,^{17} M.M. Zoeller,^{17} G. Crawford,^{18} C. M. Daubenmier,^{18} R. Fulton,^{18} D. Fujino,^{18} K.K. Gan,^{18} K. Honscheid,^{18} H. Kagan,^{18} R. Kass,^{18} J. Lee,^{18} R. Malchow,^{18} Y. Skovpen,^{18}^{†} M. Sung,^{18} C. White,^{18} F. Butler,^{19}

X. Fu,^{19} G. Kalbfleisch,^{19} W.R. Ross,^{19} P. Skubic,^{19} M. Wood,^{19} J.Fast ,^{20} R.L. McIlwain,^{20} T. Miao,^{20} D.H. Miller,^{20} M. Modesitt,^{20} D. Payne,^{20} E.I. Shibata,^{20}

I.P.J. Shipsey,^{20} P.N. Wang,^{20} M. Battle,^{21} J. Ernst,^{21} L. Gibbons,^{21} Y. Kwon,^{21} S. Roberts,^{21} E.H. Thorndike,^{21} C.H. Wang,^{21} J. Dominick,^{22} M. Lambrecht,^{22}

S. Sanghera,^{22} V. Shelkov,^{22} T. Skwarnicki,^{22} R. Stroynowski,^{22} I. Volobouev,^{22} G. Wei,^{22} P. Zadorozhny,^{22} M. Artuso,^{23} M. Goldberg,^{23} D. He,^{23} N. Horwitz,^{23} R. Kennett,^{23} R. Mountain,^{23} G.C. Moneti,^{23} F. Muheim,^{23} Y. Mukhin,^{23} S. Playfer,^{23} Y. Rozen,^{23}

S. Stone,^{23} M. Thulasidas,^{23} G. Vasseur,^{23} X. Xing,^{23} and G. Zhu^{23}

(CLEO Collaboration)

1Vanderbilt University, Nashville, Tennessee 37235

2Virginia Polytechnic Institute and State University, Blacksburg, Virginia, 24061 3California Institute of Technology, Pasadena, California 91125 4University of California, San Diego, La Jolla, California 92093 5University of California, Santa Barbara, California 93106 6Carnegie-Mellon University, Pittsburgh, Pennsylvania 15213 7University of Colorado, Boulder, Colorado 80309-0390 8Cornell University, Ithaca, New York 14853 9University of Florida, Gainesville, Florida 32611 10Harvard University, Cambridge, Massachusetts 02138 11University of Illinois, Champaign-Urbana, Illinois, 61801

12Carleton University, Ottawa, Ontario K1S 5B6 and the Institute of Particle Physics, Canada 13McGill University, Montréal, Québec H3A 2T8 and the Institute of Particle Physics, Canada

14Ithaca College, Ithaca, New York 14850 15University of Kansas, Lawrence, Kansas 66045 16University of Minnesota, Minneapolis, Minnesota 55455 17State University of New York at Albany, Albany, New York 12222 18Ohio State University, Columbus, Ohio, 43210 19University of Oklahoma, Norman, Oklahoma 73019 20Purdue University, West Lafayette, Indiana 47907 21University of Rochester, Rochester, New York 14627 22Southern Methodist University, Dallas, Texas 75275 23Syracuse University, Syracuse, New York 13244

Abstract

A search for lepton flavor violating decays of the tau into either three charged leptons or one charged lepton and two charged hadrons was performed using 2.05fb^{−1} of data collected by the CLEO-II experiment at CESR. The upper limits obtained for 22 decay branching fractions are several times more stringent than those set previously.

PACS numbers: 11.30.Fs, 13.35.Dx, 14.60.Fg

Typeset using REVTEX

∗Permanent address: University of Hawaii at Manoa

†Permanent address: INP, Novosibirsk, Russia

There is no fundamental motivation for lepton flavor conservation in the Standard Model since there is no symmetry associated with lepton family number. Therefore, searches for the lepton flavor changing processes probe basic assumptions of the theory. Many extensions of the Standard Model predict lepton flavor violation; these extensions are constrained by strict limits on neutrinoless muon decays: B(μ → eγ) < 4.9 · 10^{−11} and B(μ → eee) < 2.4 · 10^{−12} at 90% CL [1]. In some of the models there is an enhancement of the rate of neutrinoless tau decays due to mass-dependent couplings of the proposed new physics processes. This has been emphasized in recent studies of left-right symmetric models [2] and superstring models [3].

The most restrictive limits so far for neutrinoless tau decays into three charged particles were published by the ARGUS [4] and CLEO [5] collaborations. A search for the decay τ → μγ using CLEO-II data has also been published [6]. In this letter we present the results of a search for tau decays into three charged leptons:

\begin{matrix}τ^{−} → e^{−}e^{+}e^{−}, & τ^{−} → e^{−}e^{−}μ^{+}, \\ τ^{−} → e^{−}e^{+}μ^{−}, & τ^{−} → e^{−}μ^{+}μ^{−}, \\ τ^{−} → e^{+}μ^{−}μ^{−}, & τ^{−} → μ^{−}μ^{+}μ^{−}, \end{matrix}

together with a search for tau decays into one charged lepton and two charged hadrons (pions or kaons):

\begin{matrix}τ^{−} → e^{−}π^{+}π^{−}, & τ^{−} → μ^{−}π^{+}π^{−}, \\ τ^{−} → e^{−}π^{−}K^{+}, & τ^{−} → μ^{−}π^{−}K^{+}, \\ τ^{−} → e^{−}π^{+}K^{−}, & τ^{−} → μ^{−}π^{+}K^{−}, \\ τ^{−} → e^{+}π^{−}π^{−}, & τ^{−} → μ^{+}π^{−}π^{−}, \\ τ^{−} → e^{+}π^{−}K^{−}, & τ^{−} → μ^{+}π^{−}K^{−}. \end{matrix}

We also search for tau decays into one charged lepton and a neutral meson which decays subsequently into two charged hadrons:

\begin{matrix}τ^{−} → e^{−}ρ^{0}, & τ^{−} → μ^{−}ρ^{0}, \\ τ^{−} → e^{−}K^{∗ 0}, & τ^{−} → μ^{−}K^{∗ 0}, \\ τ^{−} → e^{−}K̄^{∗ 0}, & τ^{−} → μ^{−}K̄^{∗ 0}. \end{matrix}

Charge conjugate reactions are also searched for in all 22 channels.

The data used in this analysis were collected with the CLEO-II detector at the Cornell Electron Storage Ring (CESR). A detailed description of the apparatus can be found in ref. [7]. Tau leptons were produced in pairs in e^{+}e^{−} collisions. About 60% of the events were obtained at the Υ(4S) and Υ(1S) resonances while the rest were obtained at energies approximately 60 MeV below the resonances. The data correspond to an integrated luminosity of 2.05fb^{−1}, and the number of produced tau pairs, N_{ττ}, is about 1.87 · 10^{6}.

Events are selected in a 1-vs-3 topology, where the 3-prong side is a candidate for neutrinoless τ decay and the 1-prong side is an allowed τ decay into a charged particle, zero or more photons and at least one neutrino. For each candidate event we require four wellreconstructed tracks of charged particles with zero total charge. The event topology is determined by defining a plane perpendicular to each momentum vector and counting tracks on either side of the plane. Events with no track separated by more than 90 degrees from all other tracks are rejected. In ambiguous cases, where more than one such track exists, the event topology is selected by choosing the solution with the maximum angle between the 1-prong track momentum vector and the sum of momenta on the 3-prong side. Finally, events with more than one photon candidate in the 3-prong hemisphere with energy larger than 60 MeV are rejected.

Substantial background suppression comes from the lepton identification on the 3-prong side. The particle identification requirements used to select electron and muon candidates in purely leptonic channels are not very stringent. A particle is identified as an electron by requiring that the energy deposited by the particle in the CsI calorimeter, E, does not differ from its measured momentum, p, by more than 20%: 0.8 < E/p < 1.2. A particle is identified as a muon if it passes through more than three hadronic absorption lengths of the muon system and if the energy of the associated signal in the calorimeter is compatible with that expected for a minimum ionizing particle, 0.1 GeV < E < 0.5 GeV. If there are already two positively identified leptons on the 3-prong side, the muon identification requirements are relaxed to increase the selection efficiency. In such cases, the third particle is considered a muon candidate if it is not already identified as an electron, it has momentum below the threshold for effective muon detection, p < 1.5 GeV/c, and its energy deposit in the calorimeter is compatible with that of a minimum ionizing particle.

Channels containing a lepton and two hadrons in their final state suffer from a large background from copious tau decays into three pions and a neutrino, where one of the pions is misidentified as an electron or a muon. In such channels more restrictive criteria are used for lepton identification: an electron candidate is required to have 0.9 < E/p < 1.05, and ionization energy loss and time-of-flight measurements for this candidate must be consistent with the electron hypothesis whenever these measurements are available. Stricter muon identification is obtained by demanding that the penetration depth exceeds five absorption lengths.

We do not require unambiguous particle identification for hadrons. For kaon candidates we impose loose criteria on ionization energy loss and time-of-flight measurements by requiring them to be within 2.5 standard deviations of their expected values. We try all possible permutations of kaons and pions for all charged particles which are not identified as leptons to calculate kinematic parameters for tau decays into one charged lepton and two hadrons in the final state. Thus, a single event can be a candidate for more than one final state. We avoid counting the same event twice in τ^{−} → l^{+}π^{−}K^{−} channels (l stands for e or μ) by choosing the permutation with the 3-prong invariant mass closest to the tau mass. In the τ → lM channels, where M represents neutral meson ρ^{0}, K^{∗ 0} or K̄^{∗ 0}, we require the two-hadron invariant mass to be consistent with that of the corresponding meson: M_{π^{+}π^{−}} < 1.2 GeV/c^{2} and 0.7 GeV/c^{2} < M_{π^{±}K^{∓}} < 1.1 GeV/c^{2}.

The dominant background remaining after the application of the event topology and particle identification criteria is due to photon conversions in radiative Bhabha and muon pair events. This background is reduced by allowing at most one charged particle with momentum greater than 0.85 of the beam momentum and requiring the invariant mass of two oppositely charged particles, M_{e}^{+}_{e}^{−}, to be above 150 MeV/c^{2}, assuming electron masses for each particle. The remaining backgrounds are due to 2-photon processes, low multiplicity e^{+}e^{−} → qq̄ events, beam-gas events and τ decays into three hadrons and a neutrino where at least one hadron is misidentified as a lepton. In signal events there will be either one or two neutrinos on the 1-prong side which can result in a large total transverse momentum with respect to the beam direction, p_{T}. In contrast, two-photon background has a characteristically low value of the transverse momentum. We reduce two-photon background contributions by requiring p_{T} > 0.2 GeV/c. Events with neutrino emission from the 1-prong tau decay should exhibit a substantial acollinearity between the direction of the sum of the charged particles’ momenta in the 3-prong hemisphere of the event and the 1-prong momentum. The data, however, show a pronounced peak near zero acollinearity angle. This background can be eliminated by requiring the acollinearity to be greater than 3 degrees. Finally, in the rest frame of a 1-prong tau each of its decay products must have momentum less than half of the τ mass. The boost parameters from the laboratory to the tau rest frame are obtained by summing the four-momenta of the 3-prong side particles which for neutrinoless decays define the tau direction and energy. Neglecting radiative effects, the other τ in the event has an opposite momentum vector. The requirement that the 1prong momentum in the parent tau rest frame be less than 1 GeV/c considerably reduces the background. All of the above requirements do not significantly affect the efficiency of event selection. This is illustrated in Fig. 1, where the kinematical distributions for selected channels are compared with the Monte Carlo expectations for the neutrinoless decays.

The efficiencies of the selection criteria are estimated using 10 000 Monte Carlo events of each neutrinoless decay channel. The KORALB/TAUOLA program package [8] was used to simulate the tau-pair production and the decay of the 1-prong tau. Neutrinoless tau decays into three particles are assumed to follow a 3-body phase space distribution and decays into one charged lepton and one neutral meson are generated according to a 2-body phase space. The means and widths of the Breit-Wigner shapes of the meson’s mass distributions are taken from the Particle Data Group [1], and the subsequent meson decays and decays of the opposite tau are generated according to their branching fractions. Detector signals are simulated by the standard CLEO-II reconstruction program [9]. The selection efficiencies obtained are summarized in Table I.

For a neutrinoless τ decay the total energy measured on the 3-prong side, E_{3}, must be equal to the beam energy, E_{beam}, and the invariant mass of the three charged particles, M_{3}, must be equal to the τ mass. For each channel we consider a rectangular signal region in the E_{3} − E_{beam} and M_{3} variables. The size of this signal region depends on the resolution of the invariant mass and energy measurements and, therefore, varies for different channels. In particular, final state radiation degrades the momentum resolution for electrons, affecting the size of the signal region. The signal region range is also sensitive to the background level. In order to avoid any bias due to fluctuations in the data sample, we determine the size of the signal region using a criterion independent of the events near E_{3} = E_{beam} and M_{3} = M_{τ}. This criterion is based on studies of the expected limit of the decay branching fraction as a function of the signal region definition. These studies consist of extrapolating the background from the sidebands of the M_{3} vs. E_{3} − E_{beam} distributions into the signal region. For each signal region definition we calculate the expected upper limit on the branching fraction (assuming no signal present) and choose the signal region which minimizes this limit. We obtain similar bounds for all channels with one lepton and two hadrons in the final state and for practical reasons we define the same signal region for all of them: −0.14 GeV < E_{3}−E_{beam} < 0.08 GeV and 1.74 GeV/c^{2} < M_{3} < 1.80 GeV/c^{2}. We also group together all leptonic channels and define their signal regions to be −0.30 GeV < E_{3} − E_{beam} < 0.10 GeV and 1.71 GeV/c^{2} < M_{3} < 1.81 GeV/c^{2}. The channel τ^{−} → e^{−}e^{+}e^{−} requires special consideration because it has the largest radiative effects and a very low background level. For this channel we choose the limits −0.50 GeV < E_{3} − E_{beam} < 0.08 GeV and 1.66 GeV/c^{2} < M_{3} < 1.81 GeV/c^{2}.

The 3-prong invariant mass distributions of events passing all selection criteria and lying inside the E_{3} − E_{beam} limits defined above are shown in Fig. 2, together with the expected signal shapes generated by the Monte Carlo. Within the signal region, there is one event in the τ^{−} → e^{−}π^{+}K^{−} channel which is also a candidate for τ^{−} → e^{−}K̄^{∗ 0}. There is one event in the τ^{−} → μ^{−}π^{−}K^{+} channel and two events in the τ^{−} → μ^{+}π^{−}K^{−} channel. The number of observed events is consistent with the estimated background due to hadron misidentification as an electron or a muon. No candidates are found in all other channels.

We do not subtract the remaining background. Instead, using Poisson statistics for n observed events in the signal region, we obtain the corresponding branching fraction limits at 90% CL given by B = λ_{n}/2 ϵ N_{ττ} (λ_{0} = 2.30, λ_{1} = 3.89, etc.).

Since no predictive model for neutrinoless τ decays exists, we do not assign any systematic error due to model dependence. However, we emphasize that our limits depend on the assumed angular and momentum distributions of the decay particles. The main systematic errors in the analysis arise from uncertainties in our knowledge of the luminosity, track reconstruction efficiency, lepton identification efficiency and 3-prong energy and invariant mass resolutions. Combined together, they are conservatively estimated to increase the final limits by 10%. The corrected limits are summarized in Table I. For comparison, the table also shows the most restrictive upper limits obtained by previous experiments [1,4]. The limits obtained in this analysis are more stringent than those obtained previously. In addition, the limit on B(τ → eee) is the most stringent limit to date on lepton flavor violation in τ decays.

We gratefully acknowledge the effort of the CESR staff in providing us with excellent luminosity and running conditions. This work was supported by the National Science Foundation, the U.S. Department of Energy, the Heisenberg Foundation, the SSC Fellowship program of TNRLC, and the A.P. Sloan Foundation.

REFERENCES

[1] Particle Data Group, K. Hikasa et al., Phys. Rev. D 45, S1 (1992).

[2] R.N. Mohapatra, Phys. Rev. D 46, 2990 (1992); R.N. Mohapatra, S. Nussinov and X. Zhang, UMDHEP 94-30 (1993).

[3] R. Arnowitt and P. Nath, Phys. Rev. Lett. 66, 2708 (1991); J. Wu, S. Urano and R. Arnowitt, Phys. Rev. D 47, 4006 (1993).

[4] H. Albrecht et al., Zeitschrift fur Physik C 55, 179 (1992).

[5] T. Bowcock et al., Phys. Rev. D 41, 805 (1990).

[6] A. Bean et al., Phys. Rev. Lett. 70, 138 (1993).

[7] Y. Kubota et al., Nucl. Instrum. Methods Phys. Res., Sect. A 320, 66 (1992).

[8] S. Jadach and Z. Wa̧s, Comput. Phys. Commun. 64, 267 (1991); S. Jadach et al., Comput. Phys. Commun. 76, 361 (1993).

[9] Detector simulation is based on the GEANT software package: R. Brun et. al., GEANT version 3.15, CERN DD/EE/84-1.

FIGURES

FIG. 1. Kinematical distributions of variables used in the event selection procedure for the data and Monte Carlo (MC) generated events: (a) invariant mass of two oppositely charged particles assumed to be electrons, M_{e}_{+}_{e}_{−}, (b) total transverse momentum of all charged particles in the event, p_{T}, (c) acollinearity angle between the vector sum of the 3-prong momenta and the 1-prong direction, A, (d) 1-prong particle momentum after its boost to the rest system of the parent τ, p_{BOOSTED}.

FIG. 2. Distributions of the invariant mass of the 3-prong side particles, M_{3}. The expected signal shapes are shown with arbitrary normalization. The dotted lines indicate the boundaries of the signal regions used.

TABLES

TABLE I. The detection efficiencies, ϵ, and upper limits for the branching fractions at 90% CL, B, together with the most restrictive limits from previous experiments [1,4].

Decay  ϵ (%)  B (in units of 10^{−6})
channel  Previous  This analysis
τ^{−} → e^{−}e^{+}e^{−}  20.4  13  3.3
τ^{−} → μ^{−}e^{+}e^{−}  19.6  14  3.4
τ^{−} → μ^{+}e^{−}e^{−}  19.9  14  3.4
τ^{−} → e^{−}μ^{+}μ^{−}  18.8  19  3.6
τ^{−} → e^{+}μ^{−}μ^{−}  19.4  16  3.5
τ^{−} → μ^{−}μ^{+}μ^{−}  15.9  17  4.3
τ^{−} → e^{−}π^{+}π^{−}  15.5  27  4.4
τ^{−} → e^{−}π^{−}K^{+}  14.6  58  4.6
τ^{−} → e^{−}π^{+}K^{−}  14.9  29  7.7
τ^{−} → e^{+}π^{−}π^{−}  15.5  17  4.4
τ^{−} → e^{+}π^{−}K^{−}  15.1  20  4.5
τ^{−} → μ^{−}π^{+}π^{−}  9.1  36  7.4
τ^{−} → μ^{−}π^{−}K^{+}  7.4  77  15
τ^{−} → μ^{−}π^{+}K^{−}  7.8  77  8.7
τ^{−} → μ^{+}π^{−}π^{−}  9.8  39  6.9
τ^{−} → μ^{+}π^{−}K^{−}  7.7  40  20
τ^{−} → e^{−}ρ^{0}  16.2  19  4.2
τ^{−} → e^{−}K^{∗ 0}  10.7  38  6.3
τ^{−} → e^{−}K̄^{∗ 0}  10.5  —  11
τ^{−} → μ^{−}ρ^{0}  11.9  29  5.7
τ^{−} → μ^{−}K^{∗ 0}  7.2  45  9.4
τ^{−} → μ^{−}K̄^{∗ 0}  7.8  —  8.7