James Nelson Williams was born at Waimate mission station inland from the Bay of Islands, New Zealand, probably on 22 August 1837, the third son of Jane Nelson and her husband, William Williams, a missionary and later bishop of Waiapu. He was taken by his parents to the Church Missionary Society station at Tūranga (Gisborne) in 1839. An 'active little fellow not deficient in abilities', James grew up in a close family circle and was sent to the school at St John's College in Auckland in 1846. Due to what his father considered a 'very sad deficiency in almost every department of the school', he was taken home in 1848 'behindhand in almost everything.' In 1850 he went to England with his parents for 'the sake of…a little English education'.
On their return to New Zealand in 1853, James helped his father by ploughing and fencing at Waerenga-a-hika, the new site of the mission station, until it was decided he should take up sheepfarming in Hawke's Bay. In February 1857 he went to Te Aute to work with his cousin and future brother-in-law the Reverend Samuel Williams. With his father's financial assistance he then acquired 20,000 acres at Kererū, where, by 1860, he ran 3,200 sheep. After entering into partnership with Colonel Jasper Herrick in 1865 he was soon able to repay his father with interest.
On 22 January 1868 at Lower Hutt James Williams married a Wairarapa pastoralist's daughter, Mary Margaret Beetham. They were to have four sons and three daughters, two of whom were to cement their father's business partnerships by marrying sons of W. R. Russell and William Nelson. James's closest partner, however, was Samuel Williams, who secured on his behalf two out of 12 shares in the Heretaunga block when it was leased from its Māori owners by Thomas Tanner in 1864. With his shares he acquired 2,462 acres of land which he named Frimley; a further acquisition brought the property to 3,405 acres. He moved there after marrying and sold Kererū in 1875.
Two-thirds of Frimley was swampy land which had to be drained and cleared of flax and scrub before it could be ploughed and cropped. As he later explained in Permanent pastures (1906), scarcely anyone then understood this novel type of farming. Indeed, he himself was one of the few to succeed. Like other agriculturalists on the plains, he also had to contend with dock, thistles, the small bird nuisance and the Ngaruroro River in flood. With the help of managers, gardeners and labourers, he established a fine homestead block and a Clydesdale stud, and won prizes for his oats, barley and potatoes at Hawke's Bay Agricultural and Pastoral Society shows. He was president of the society in 1875, 1884 and 1899.
Williams possessed the talents needed to succeed in bold, innovative investments and enterprises. He straightened out the confused and complicated accounts when Tanner's syndicate purchased the Heretaunga block in 1870. He became the Williams family's financier, and was a director of the Northern Investment Company of New Zealand, which channelled Scottish capital into Hawke's Bay. He was also a shareholder in and director of practically every scheme to advance the progress and welfare of the province.
Williams continued to lease, purchase, break in and dispose of large areas of land for sheep and cattle grazing and breeding: the Apley portion of Rissington was sold in 1879, Mangakurī was sold to Samuel Williams in 1899, and Edenham was divided up between members of his family sometime before his death in 1915. Williams leased some 39,000 acres in the Waipiro Bay district in 1883, developing it and handing it back to the Māori owners in 1915 when the lease finally expired. After the township of Hastings was laid out in 1873, he took a proprietorial interest in its advancement, buying 21 town sections for £725 in 1875. He subdivided and sold adjoining Frimley land in suburban and farm lots at the turn of the century, and invested in ferro-concrete commercial buildings in 1911–12.
In 1880 Williams partnered William Nelson in building a boiling-down works, which opened at Tomōana in 1881. He remained a shareholder after the company was partly sold in 1882. On a trip overseas in 1890 Williams visited orchards near Niagara Falls and Bordeaux, and a fruit township supporting 30,000 people in California. The information he gathered enabled him to establish commercial fruit-growing in Hawke's Bay. At Frimley, about 1892, he started with a two-acre vineyard and a 10-acre orchard. He then offered about 1,800 acres to the Liberal government for closer settlement. The government, however, purchased only 1,133 acres which it opened up for selection in 1899 as the Mahora settlement.
Williams's Frimley orchard consisted of 60 miles of peach trees. In 1904 he opened a canning factory under the management of Basil Jones from California. Its workforce of 70 expanded to a peak of 250 in 1909–10. Williams was a paternalistic employer who provided good facilities for his workers, including accommodation, transport and recreation. With Thomas Horton he pioneered the export fruit trade, and was founding chairman of the Hawke's Bay Fruit Growers' Association which floated a company to manage local fruit marketing. The boom in commercial fruit-growing and canning was short-lived. Severe frost in 1911 and bad management led to the closure of the canning factory. Williams was ageing and in failing health. He went to live with a married daughter in Havelock North, where he died on 11 June 1915. Mary Williams had died in London on 25 February 1903.
Williams was remembered as a kind, generous man, widely respected because he personally worked his land and provided employment for many people. Since the Heretaunga Road Board first met in his Frimley woolshed in 1871, he had given long, valuable service on the local bodies that administered the Frimley district. He was a lay member of the diocesan synod and an active churchwarden at St Matthew's Church, Hastings. He was also a member of the local acclimatisation society, the cricket association, the Hawke's Bay Club, and of the Hastings Club, where W. A. Bowring's portrait of him hangs. Williams gave a ward to Napier Hospital and 21 acres to Hastings for Cornwall Park. When the Frimley homestead was burnt down in 1951 his daughter, Elsie, and sons, Arnold and Heathcote, donated the 47-acre Frimley Park to the borough in memory of their parents. The magnificent trees and gardens in these parks are a constant reminder of a pioneer agriculturalist, pastoralist, orchardist and beautifier.