SECURITIES AND EXCHANGE COMMISSION

Washington, D.C. 20549

SCHEDULE 13D

Under the Securities Exchange Act of 1934

(Amendment No. 4)

The Chalone Wine Group, Ltd.

(Name of Issuer)

Common Stock, no par value

(Title of Class of Securities)

157639105

(CUSIP Number)

Thomas J. Mullin, Esq.

Executive Vice President and

General Counsel

Constellation Brands, Inc.

370 Woodcliff Drive

Suite 300

Fairport, New York 11450

(585) 218-3650

With a copy to:

Richard F. Langan, Jr., Esq.

Nixon Peabody LLP

437 Madison Avenue

New York, New York 10022

(Name, Address and Telephone Number of Person Authorized to Receive Notices and Communications)

August 20, 2004

(Date of Event which Requires Filing of this Statement)

If the filing person has previously filed a statement on Schedule 13G to report the acquisition that is the subject of this Schedule 13D, and is filing this schedule because of § 240.13d-1(e), 240.13d-1(f) or 240.13d-1(g), check the following box ¨.

Note: Schedules filed in paper format shall include a signed original and five copies of the schedule, including all exhibits. See § 240.13d-7(b) for other parties to whom copies are to be sent.

The information required in the remainder of this cover page shall not be deemed to be “filed” for the purpose of Section 18 of the Securities Exchange Act of 1934 (“Act”) or otherwise subject to the liabilities of that section of the Act but shall be subject to all other provisions of the Act (however, see the Notes).

SCHEDULE 13D

This statement constitutes Amendment No. 4 to the Schedule 13D filed with the Securities and Exchange Commission (the “SEC”) on behalf of Constellation Brands, Inc., a Delaware corporation (“Constellation”) on May 17, 2004 (the “Schedule 13D”), with respect to the common stock, no par value (the “Common Stock”), of The Chalone Wine Group, Ltd., a California corporation (the “Issuer”). Capitalized terms not otherwise defined herein shall have the respective meaning ascribed thereto in the Schedule 13D.

Other than as set forth herein, there has been no material change in the information set forth in the Schedule 13D.

Item 4. Purpose of the Transaction.

Item 4 of the Schedule 13D is hereby supplemented as follows:

As described more fully in Amendment No. 3 to the Schedule 13D, DBR delivered to the board of directors of the Issuer a Second Proposal Extension Letter attached thereto as Exhibit 6, extending the offer contained in its Proposal Letter to August 20, 2004.

On August 20, 2004, DBR extended the offer contained in its Proposal Letter (the “Third Proposal Extension Letter”) until August 27, 2004. The Third Proposal Extension Letter was acknowledged by Constellation and Huneeus. The remaining terms of the Proposal Letter remain in full force and effect. The Third Proposal Extension Letter is annexed hereto as Exhibit 7 and incorporated herein by reference.

Item 7. Material to Be Filed as Exhibits.

Item 7 of the Schedule 13D is supplemented as follows:

Signature

After reasonable inquiry and to the best of the undersigned’s knowledge and belief, the undersigned certifies that the information set forth in this statement is true, complete and correct.

Date: August 20, 2004

EXHIBIT INDEX