In a recent article, Kim Bayliss described a scheme for classifying AI inventions. In further articles (part I and II), Kim described how applications should be drafted to take account of the nature of the AI invention, in the light of the EPO practice. This article uses that scheme as the basis for looking in detail at how the patent application process at the European Patent Office (EPO) handles some AI inventions.
To summarise very briefly, the recent article said that AI inventions can be classified as “Applied AI”, “Core AI”, or “Hardware”, and made the point that patent applications for AI inventions in different categories will need to be drafted differently, and will be examined differently by the EPO.
EP-3319016-B is a European patent that has recently been granted to Raytheon Corporation. The invention relates to control systems using deep reinforcement learning. The application describes in detail the deep reinforcement learning process, in which sensor outputs are used to generate a reward, which is based on how closely the sensor outputs approach a desired state of the system. A deep neural network (DNN) then determines an action, which can be executed in order to increase the reward value. Based on the resulting new reward value, a policy parameter of the control system can be updated, and the updated policy parameter can be used to determine subsequent actions.
The application mentions many different fields in which such a deep reinforcement learning process can be applied, but it is described in the application specifically in the context of a coldspray or additive manufacturing process, in which a workpiece can be refurbished if a defect is detected. The speed and angle of a spray nozzle can be adjusted, in order to control the way in which material is deposited onto the surface of the workpiece. The purpose of the control system is to optimize the control of the nozzle, and hence the deposition of the material, in order to obtain the desired finish of the refurbished workpiece.
The application describes a particular example, in which a component refurbishing system contains a physical system and also a simulation of that physical system. In that example, the deep reinforcement learning process is used in the simulation environment, in order to learn control policies that can be applied in the physical component refurbishing system.
The main claims of the application as filed define the deep reinforcement learning process, with its use in a coldspray application only being defined in subclaims.
As such, using the terminology from the other article mentioned above, the application as filed presented the invention as a “Core-AI” invention, defining a new AI process that could be applied to many different scenarios and different data types. As such, the application contained claims without any limitation to any particular field of use.
In the first examination report, the EPO examiner raised various objections.
One of these objections was that, although the process is apparently intended to be performed on a processing system, this was not specified in claim 1. This was expressed as an objection that the claim lacked clarity, but the examiner could perhaps also have objected that, if the claim is not limited to a processing system, then the claim could be said to relate to a mathematical method or a mental act.
A second objection raised by the EPO examiner was that claim 1 as filed lacked an inventive step. In raising this objection, the examiner relied on a prior art document that relates to control of a robot arm, but the examiner made the point that “the general methodology for learning control policies … can be translated into many different settings”, and hence that the broad claim 1 cannot be considered inventive.
However, despite this, the EPO examiner did say that the application could be found to be allowable, if the claims were limited to the specific application of the claimed deep reinforcement learning process to the described nozzle spraying application, and to the use of the deep reinforcement learning process in the simulation environment.
There were several further rounds of examination to arrive at a final version of the claims, but eventually the applicant proposed claims that include these limitations, and that the EPO examiner found to be allowable. This process illustrates an aspect of examination at the EPO that often arises, namely that, because of the EPO practice on added subject matter, it can often be difficult to write claims that are based on the description and/or drawings that comply with the EPO practice, without being unduly limited to a specific embodiment of the invention.
The patent has now been granted, with main claims that are limited to one application of the deep reinforcement learning process that is described in the application. The claims specify that the action that is performed controls the “speed and [the] angle of the nozzle over multiple applications of material in layers”. The claims further specify that the control process “tak[es] into account a weighted sum of a time to complete a pass and a penalty on angular rate”.
Finally, the claims include the limitation that the control policy, using the deep reinforcement learning process, is developed in a simulation environment.
Thus, using the terminology from the other article mentioned above, the patent as granted now relates to an “Applied-AI” invention, where the claims define the manner in which the AI algorithm is used, because the EPO examiner did not see anything inventive in the AI algorithm itself.
In this case, it was only possible to get a patent granted because the application as filed contained a clear use-case including details of a specific field in which the deep reinforcement learning technique could be used. This use-case was then used as basis to amend the claims and convert the (unallowable) Core AI invention into an allowable Applied AI invention. This clearly illustrates some of the pitfalls in trying to obtain broad protection for Core AI inventions in Europe, and illustrates the importance of including detailed and commercially useful use-cases that can be used as fall-back positions in the detailed description.
This is for general information only and does not constitute legal advice. Should you require advice on this or any other topic then please contact firstname.lastname@example.org or your usual Haseltine Lake Kempner advisor.