SECURITIES AND EXCHANGE COMMISSION

Washington, D.C. 20549

SCHEDULE 13D

Under the Securities Exchange Act of 1934

(Amendment No. 5)

The Chalone Wine Group, Ltd.

(Name of Issuer)

Common Stock, no par value

(Title of Class of Securities)

157639105

(CUSIP Number)

Thomas J. Mullin, Esq.

Executive Vice President and

General Counsel

Constellation Brands, Inc.

370 Woodcliff Drive

Suite 300

Fairport, New York 11450

(585) 218-3650

With a copy to:

Richard F. Langan, Jr., Esq.

Nixon Peabody LLP

437 Madison Avenue

New York, New York 10022

(Name, Address and Telephone Number of Person Authorized to Receive Notices and Communications)

August 27, 2004

(Date of Event which Requires Filing of this Statement)

If the filing person has previously filed a statement on Schedule 13G to report the acquisition that is the subject of this Schedule 13D, and is filing this schedule because of § 240.13d-1(e), 240.13d-1(f) or 240.13d-1(g), check the following box   ¨.

Note: Schedules filed in paper format shall include a signed original and five copies of the schedule, including all exhibits. See § 240.13d-7(b) for other parties to whom copies are to be sent.

The information required in the remainder of this cover page shall not be deemed to be “filed” for the purpose of Section 18 of the Securities Exchange Act of 1934 (“Act”) or otherwise subject to the liabilities of that section of the Act but shall be subject to all other provisions of the Act (however, see the Notes).

This statement constitutes Amendment No. 5 to the Schedule 13D filed with the Securities and Exchange Commission (the “SEC”) on behalf of Constellation Brands, Inc., a Delaware corporation (“Constellation”) on May 17, 2004 as previously amended by Amendments No. 1 through 4 thereto, (the “Schedule 13D”), with respect to the common stock, no par value (the “Common Stock”), of The Chalone Wine Group, Ltd., a California corporation (the “Issuer”). Capitalized terms not otherwise defined herein shall have the respective meaning ascribed thereto in the Schedule 13D.

Other than as set forth herein, there has been no material change in the information set forth in the Schedule 13D.

Item 4. Purpose of the Transaction.

Item 4 of the Schedule 13D is supplemented as follows:

As described more fully in Amendment No. 4 to the Schedule 13D, DBR delivered to the board of directors of the Issuer a Third Proposal Extension Letter, attached thereto as Exhibit 7, extending the offer contained in its Proposal Letter to August 27, 2004.

On August 27, 2004, DBR extended the offer contained in its Proposal Letter (the “Fourth Proposal Extension Letter”) until September 3, 2004. The Fourth Proposal Extension Letter was acknowledged by Constellation and Huneeus. The remaining terms of the Proposal Letter remain in full force and effect. The Fourth Proposal Extension Letter is annexed hereto as Exhibit 8 and incorporated herein by reference.

Item 5. Interest in Securities of the Issuer.

Item 5 of the Schedule 13D is supplemented as follows:

Constellation has been advised by DBR that on August 21, 2004, DBR acquired 1,033,363 additional shares of Common Stock of the Issuer upon the maturity of a convertible subordinated promissory note of the Issuer held by DBR. Accordingly, based on information provided to Constellation by DBR, DBR now owns 6,589,466 shares of Common Stock, representing approximately 49% of the outstanding Common Stock. As a result, to the knowledge of Constellation, members of the group (consisting of Constellation, DBR and Huneeus) beneficially own an aggregate of 6,589,466 shares of Common Stock, representing approximately 49% of the total number of shares of such class.

Item 7. Material to Be Filed as Exhibits.

Item 7 of the Schedule 13D is supplemented as follows:

Signature

After reasonable inquiry and to the best of the undersigned’s knowledge and belief, the undersigned certifies that the information set forth in this statement is true, complete and correct.

Date: August 30, 2004

EXHIBIT INDEX