Bad Machine Data: Fixing Data Quality Before It Reaches Power BI

iFAKT GmbH - Bad Machine Data - Fixing Data Quality Before It Reaches Power BI

00:00:00 Speaker: Which number should you trust? Power BI can calculate the right answer from the wrong events. So we're going to trace those events back toward the machine, work through the checks they need, and keep uncertainty attached to the numbers people use to make production decisions. The first useful distinction is simple. A correct calculation doesn't prove that the underlying production record is correct. Let's take a hypothetical packaging line. Packs pass through a filler, move into a container, and leave through a final count point before the finished cases reach the pallet area. During the night shift, the katana stops several times. The operator clears the jams and restarts production, while the collection system records some of those changes, but misses the stop event for one of the longer interruptions. That leaves a gap in the machine's recorded story. Depending on how the report builds running time, that gap could leave the line recorded as running through an interval when no packs moved at all. Meanwhile, the connection drops and later recovers. Some count events arrive again after the reconnect, and the reporting process treats those repeated deliveries as fresh production because it can't tell them apart. There's a pressure signal in the same data feed. Its readings look reasonable to someone scanning a column of values, but the record doesn't clearly establish which unit those values represent when they reach the report. Nobody needs to invent a broken dashboard here. The calculations can follow their instructions exactly, while the events feeding those calculations describe a shift that doesn't match the one the operator worked. Now put yourself in. That morning meeting, the operator remembers clearing a jam near the end of the shift, but the production report assigns most of that period to running time and shows output above expectations. Someone opens the machine historian the system holding recorded process history. It contains a pressure change around the interruption. Yet that doesn't explain why the output report counts more completed production than the operator expects to hand over. Each person brings a different piece of evidence. The operator describes the physical interruption, the historian holds recorded signal values, and the shift report combines selected events into totals that management can act on. I wouldn't declare a winner from that alone. I'd ask which physical event each record claims to describe, because those records may disagree without any of the people using them misunderstanding their own system. Start with the missing stop. If the line appears to run longer than it actually ran. A supervisor might investigate slow production during running time rather than the interruption that deserves attention. The repeated count creates a different problem. A planner might believe more finished cases are available for an order, then change the next shift sequence around stock that the line never produced. The pressure reading complicates the maintenance discussion. If its meaning remains unclear, an engineer could spend time chasing an apparent process fault, while the actual fault sits in how the measurement reaches the report, those consequences don't stay inside the data team. They reach the shift leader, assigning people the planner, deciding which order runs next, and the technician trying to work out which machine needs attention first. Notice how the disagreement changes the meeting itself. Instead of discussing how to recover production, people spend their time defending the source they know best and questioning numbers they can't connect to the floor. That's a reasonable response to conflicting evidence. When operators can point to a stop they dealt with, but the report gives them no account of it. Asking them to trust the report won't settle anything. We'll keep this packaging line as our working example. Each fault gives us a separate question to follow upstream. Rather than treating every disagreement as one broad problem called bad data for the missing stop, we're asking what the record leaves out for the repeated count and unclear pressure reading. We're asking whether the recorded events describe distinct production, and whether the measurement means what its users think. But before choosing checks, I'd ask one more question what decision are we trying to support with this signal? And what could someone do wrong if the value, timing or meaning doesn't hold up, I'd set the quality target around the decision. A signal can support a shift review quite well and still lack the timing detail. An engineer needs to explain a brief machine fault. Take the packaging lines pressure measurement for a shift review. You might need a reliable view of pressure across operating periods, with enough coverage to judge whether the process stayed within its agreed range. Fault diagnosis asks for something different if an engineer needs to know whether pressure fell before the partner stopped, the readings need enough time detail to establish that sequence without guessing between widely spaced observations. Production planning brings another set of needs. A planner deciding whether to release the next order needs a sufficiently current account of available capacity, rather than a detailed history that ends before the latest interruption. Those users can share a source without sharing a quality threshold. I'd separate the requirements before anyone chooses a single definition of good data, and applies it to every report, investigation, and planning process in the plant. Accuracy is one part of that discussion. Ask how far the recorded measurement can differ from the physical quantity. Before someone would reach a different conclusion about the process, or choose a different action for pressure, that limit needs engineering input. A reading used to compare broad operating trends might tolerate an error that would make it unsuitable for judging whether the process crossed a narrow operating limit. Completeness needs the same care. A shift report might remain usable with a small declared gap, while an investigation of a short fault could lose its entire basis if that gap covers the fault itself. Freshness asks how old the information can be. A completed shift review can use settled historical records, but a planner deciding what to run now needs to know whether the reported machine condition still holds. Timing is a separate question from freshness. Two readings might arrive promptly, yet describe events with too much uncertainty in their timestamps to establish which physical change came first during a machine interruption. That distinction changes what you ask engineering to approve. Instead of asking whether timestamps look acceptable. Ask how much uncertainty the investigation can tolerate before the order of events becomes unclear and the diagnosis becomes doubtful. I wouldn't set those limits alone. Production needs to explain the action people will take, and engineering needs to explain what the equipment and measurement chain can actually support under normal operating conditions. Then test the requirement against a real decision. If the planner accepts information that several minutes old, ask what should happen when a machine stops during those minutes and an order release still depends on availability. The answer needs an operating rule. Perhaps the planner checks the current condition before committing the order, or the planning process treats information beyond an agreed age as insufficient for that decision. Some limits should stop a calculation where a missing operating period could change a performance assessment. The report may need to withhold that assessment rather than publish a precise number with weak support. Other limits can trigger a warning if a shift review still supports its intended discussion. Despite a known gap, production can agree to use it with the gap clearly stated and its effect explained. Machine control sits outside that reporting agreement. A cloud reporting path through fabric and power BI shouldn't become the mechanism responsible for a safety response or a time critical machine action that needs guaranteed local behavior. Those functions need their own engineered systems and approved safeguards. Reporting can help people review how those systems behaved, but it's delay, availability and processing choices mustn't become hidden dependencies of the machine's protective functions. There's also a cost to demanding more detail, faster collection and tighter checks consume resources. So I'd ask which decision improves before asking a plant team to support a stricter requirement across every signal, write the agreed limits in plain language for each intended use. State the acceptable measurement error the coverage needed, how old the information may be, and how precisely events need to be timed. Include what happens when a limit fails. That gives production and engineering a shared basis for deciding whether a signal can support an action, needs a warning, or must remain out of the calculation. Now that agreement needs to travel with the signal. The next step is to make its meaning and handling rules explicit enough that each system receiving it can apply the same expectations. I put that agreement into a data contract. That's a written definition of what a signal means, how its records are formed, and what receiving systems should do when those records don't meet the agreement. The word contract can sound heavier than the work involved. For one pressure signal, you can start with a short, versioned record that both the control's engineer and the person building the data pipeline can understand. Start with identity, because tag names alone can mislead you. A name like discharge Pressure needs a stable asset reference and a named measurement point, so nobody mistakes pressure at the filler inlet for pressure farther downstream. Keep that identity separate from the current network address. If someone replaces a gateway or moves a tag into another folder, the receiving system still needs to know whether it represents the same physical measurement point. Then define the value itself. State whether the field contains a whole number, a decimal measurement or a coded state, and specify the engineering unit that gives the number its physical meaning. Precision belongs in that definition, too. If the source reports pressure to one decimal place, the contract shouldn't imply that extra decimal places added during processing represent detail. The instrument actually measured at the machine include the quality status supplied by the source, where the equipment or collection interface reports that a value is doubtful, unavailable or otherwise unsuitable. That status needs a defined place alongside the value in the record. An absence status needs an agreed meaning as well. If an older source can't provide a quality indication, record that limitation explicitly, rather than letting each receiving system decide that no status must mean good. Next, describe. When the source sends information, a signal reported at a fixed interval creates a different expectation from one sent only when its value changes, even when both use the same message format. Be specific about change based behaviour. Record whether the source sends every detected change applies a threshold before sending or issues a periodic update, even when the measured value stays within that threshold, a timestamp attached when the source observes a value. Means something completely different from one attached when a collector reads it. The contract needs to spell out which meaning each record carries. You also need to state the expected delivery pattern. That means saying whether records can arrive in batches, whether reconnects can bring older records, and which fields a receiver can rely on to interpret those deliveries. Handling rules belong right beside those definitions. If a required unit or asset reference disappears, the agreement should say whether the record stays usable for any purpose and which users have to wait for the missing information. Someone has to approve changes to the signals, meaning a named receiving system owner needs responsibility for checking how those changes affect the systems that consume the data. Consider a change to the final count point. If a machine program switches from counting individual packs to counting cases, keeping the old tag name doesn't preserve the meaning that downstream users previously agreed to. That change needs a new contract version record when the new meaning takes effect, who approved it, and how a receiving system can tell which definition applies to each record it gets. Make the version available to the receiving process. That might mean carrying it in the message, or keeping a controlled mapping with effective dates, as long as the relationship stays clear. When someone reviews an older record, keep a history instead of replacing the old definition. An engineer investigating last month's production needs the agreement that applied then not a document edited yesterday to describe the machine after its latest program change. Ask them to walk through an actual message and explain where each field comes from, including fields that are gateway supplies rather than the machine itself. That review often exposes assumptions before they become dependencies. You might discover that the expected unit exists only in a project node, or that nobody has defined what happens when a requested field can't be read. With the contract drafted, you can check what the machine actually exposes and whether the collection path preserves the meaning you've just agreed to at the machine. I'd begin with the logic behind the signal. The Programmable Logic Controller or PLC runs the machine program, and that program helps explain what a reported bit or number actually represents. During production. A tag called running needs that check. It might indicate that the drive has permission to run, that a motor command is active, or that the machine sequence has entered its automatic operating step. Those conditions describe different things on the packaging line during a jam. A run command might remain active, even though packs have stopped moving through the carton and the operator has started clearing the obstruction. Ask the controls engineer to trace the bit. Follow it back through the program until you understand which conditions set it, which conditions clear it, and whether it holds its value between those changes. A part complete pulse might remain active for one program cycle. That means an external collector reading less often can miss that pulse, even while its connection works normally. The collection method has to match that behavior. Depending on what the equipment supports, the controls team might expose a retained event, a latched indication, or another approved record that survives long enough for collection. An interface can carry useful evidence beyond the value. Open platform communications unified architecture, usually called OPC, UA can expose timestamps and status codes alongside machine data where the server and collection path support them. Read those status codes rather than discarding them. A return number can arrive with a status indicating uncertainty or a failed source read. So extracting only the numeric field can remove the warning attached to it. Check timestamp behavior against the actual server. A source timestamp can describe when the data source assigned the value, while a server timestamp describes timing at the server. But implementation details still need verification. The sensor updates its output. The controller reads that input and executes its logic. The gateway requests or receives data. The cloud receives the resulting message. Each step has its own pace. A fast gateway read can't create new physical observations if the sensor updates more slowly, and a fast sensor doesn't help if the controller exposes only an occasional snapshot, trace that sequence for the pressure reading. Ask when the instrument refreshes its output, when the controller reads it, and whether any filtering or averaging occurs before the value reaches the collection interface. A filtered value answers a different question from an instantaneous reading. If the controller smooths brief pressure changes before exposing the tag, an engineer reviewing that tag needs to know which process changes the smoothing could conceal. Suppose in a hypothetical line, the collector reads machine state every five seconds while the cartoner stops and restarts entirely between two successive reads. Both reads can correctly report running. The missing interruption comes from the observation method because neither read occurred during the stop, rather than from a message disappearing somewhere between the gateway and the cloud. Changing the report won't recover that interruption. You'd need a source record that captures the transition, or a collection method with enough time resolution for the stops you're trying to measure within the equipment supported limits. Test that behaviour under controlled conditions with the controls team. Compare a known machine transition with what the controller exposes and what the collector records. Using an approved test that doesn't disrupt production or bypass safeguards include brief transitions as well as long ones. A test with a long stop can pass while shorter stops remain invisible, so the test should reflect the event durations the intended analysis needs to distinguish. Record the limits you find at the source. If the collection path can't resolve a short interruption, that limit belongs in the signals documented capability rather than becoming an unexplained disagreement in the shift report. One is an expected reading that never reaches the receiving system. The other is a physical event that the source or collection method never captured. Keep those separate when investigating a gap before deciding how to handle an absent value. Establish whether you expected a record at that moment and whether the equipment could have produced it. A blank pressure reading isn't a pressure of zero. If a report fills that blank with zero, it turns an absence of evidence into a claim that pressure disappeared, which could send maintenance toward the wrong fault. Separate the conditions before choosing a replacement value. A record that never arrives. A record containing an explicit null and a collector reporting a failed read. Each. Provide different evidence about why a measurement isn't available. An explicit null tells you that a field contains no value. Its cause still needs an explanation, because the source might use it for an unavailable measurement and inactive measurement, or a field that doesn't apply during that operating mode. A failed read tells you something else. The collector attempted to obtain a value and couldn't so preserve that result. Rather than publishing a numeric substitute that looks like a successful pressure measurement. A genuine zero needs to remain a zero if the instrument successfully measures zero within its supported range. Excluding that reading as missing. Would remove a physical condition that an engineer might need to investigate. For our packaging line, I'd make these distinctions testable give the receiving process examples of an absent record, an explicit null, a failed read, and a valid zero. Then check that each produces the intended result. For a fixed interval signal, expected coverage comes from the agreed collection interval and the periods during which collection should operate, rather than from every minute on the shift calendar. Machine mode can change that expectation. If pressure collection intentionally stops during an approved shutdown, that interval shouldn't count as an unexplained loss, provided the shutdown period itself has a reliable recorded basis. Don't let missing mode information excuse missing pressure data when you can't establish whether a collection should have operated, keep the expectation unresolved. Instead of assuming the machine entered a mode that conveniently removes the gap. Change based reporting needs a different test. If the source sends a measurement only after it changes beyond an agreed threshold. Silence between messages doesn't automatically mean that expected readings have disappeared from the feed count. Expected behavior, not imaginary samples. You can't judge a change based feed by demanding one record each second, unless the collection agreement also requires a periodic message at that rate, regardless of process changes. That leaves a practical limit on what silence tells you without another agreed indication of collection activity, an unchanged process and a broken collection path can produce the same absence of measurement messages at the receiver. For a shift review, you might publish the pressure average for observed operating periods, while stating how much of the relevant period lacks usable measurements. Name that average carefully. An average of available readings describes the observed data, and it doesn't automatically represent the whole shift. If the missing period could contain different conditions from those you captured. Be careful with unevenly spaced readings, too, giving every received value. Equal weight can distort a time based average, so define the calculation around the signals reporting behavior and the duration each observation can reasonably represent.

00:17:11 Speaker: Some users can accept a bounded estimate for a short gap in the slowly changing process. Engineering might approve interpolation between valid readings with a maximum gap length and explicit conditions that rule out its use. Keep the estimates separate from the measurement, preserve the missing interval, and label the inserted values as estimates so a later user can exclude them without having to reconstruct which numbers the instrument actually supplied. Other gaps should block the result if the missing pressure interval covers the interruption under investigation, filling it with a smooth estimate would create the process behavior you're supposed to be trying to establish. Automatic gap filling can hide faults through apparent completeness. A report with no blanks may contain less trustworthy evidence than one. That leaves a clear unknown, especially when the filled interval carries the decision. I'd agree on a handling policy for each intended use state when an unknown remains visible, when a bounded estimate is allowed, and when insufficient coverage prevents the calculation from supporting the intended action. Suppose the gateway keeps sending pressure messages throughout the cartoon. Interruption. Every delivery reaches the receiver on schedule, but the gateway has lost contact with the instrument and keeps sending the last value it successfully read. If you only check delivery, the connection looks healthy. To judge whether that pressure still describes the machine, you need evidence that the measurement path continues to obtain observations from the source, not just send messages. I'd start by checking where the repeated value comes from. Some collection systems keep a cached value after a read fails, and you need to establish whether they preserve its original timestamp and change its quality status. That behavior determines what the receiver can detect. If the gateway replaces the timestamp with the current time on each send, it can make an old observation appear current unless another field preserves the source. Observation time. Ask what has to work for that heartbeat to advance. A gateway process can remain alive while its connection to the controller fails, so a gateway heartbeat alone can't prove that the pressure measurement is still being acquired. Source timestamps provide another piece of evidence where their behavior is verified. You can compare the time of the latest source observation with the time you're evaluating the signal and assess the observations. Age check sequence progress separately from timestamp progress. If the source assigns a sequence number to each new observation, an advancing sequence can help establish that acquisition continues even when consecutive pressure values happen to match the location of that sequence. Number matters. A number assigned to outgoing gateway messages proves that the gateway creates messages, while a number assigned during source acquisition can provide evidence about activity closer to the measurement. That's where process behavior enters the investigation on our hypothetical packaging line. Suppose pressure normally changes when a particular valve opens and the valves confirmed. Operating state changes while the pressure reading remains exactly fixed. That creates a reason to question the pressure signal. It doesn't establish a failed sensor by itself, because the expected pressure response depends on the pipework operating conditions and whether the valve actually moved as intended. Use related signals to test that expectation, a second pressure measurement or a flow reading might help distinguish a stable process from a frozen measurement, provided engineering confirms what relationship should hold under that operating condition. An unchanged value can also be entirely reasonable during steady operation. A regulated pressure may remain within the instrument's resolution for a long period, so a simple, unchanged value timer could generate alarms on healthy equipment. I'd make the stuck value rule conditional on operating behavior, define when variation should occur, how much variation the instrument can resolve, and which machine conditions make an unchanged reading suspicious rather than simply steady. Keep the finding narrower than the evidence. A rule might report that pressure failed to respond during a confirmed operating change, leaving investigation to determine whether the fault lies in the instrument, its input channel or the process. Once that limit expires, change the signal's status. If you retain its value for reference, call it the last known pressure and carry its observation time and age so nobody mistakes it for a current measurement. Recovery also needs evidence from the source require a successful new observation under the agreed acquisition behavior before clearing the stale status, rather than clearing it as soon as the gateway starts delivering messages again. Clock errors can reverse the apparent order of a fault on our packaging line. A pressure drop recorded by one device could appear after the katana stopped recorded by another, even though the physical pressure drop occurred first. That changes the diagnosis. An engineer could treat low pressure as a result of the stop, when the actual sequence points toward pressure loss as something that needs investigation before the stop, start by naming the times you're comparing. Event time describes when the source records the occurrence or observation, while Gateway Time describes a step at the collector and ingestion time records arrival at the receiving platform. Those times answer different questions when the gateway assigns a timestamp because the machine supplies none, identify it as a collection time rather than claiming it records the precise moment of the physical change. Preserve that distinction through the pipeline. Ingestion time can help investigate delivery delay, but subtracting event time from ingestion. Time mixes transport delay with any disagreement between the source clock and the receiving clock. Clock jumps behave differently. A restart, manual adjustment or synchronization correction can move a clock abruptly, leaving records that appear to move backward or skip a period even while the equipment continues operating. Check startup behavior as well. Some equipment begins with an incorrect date until it reaches a time source, and records created during that interval need an explicit timing limitation rather than an assumed correction. Use coordinated Universal Time or UTC for shared event timestamps. Keep the plant's local time zone and shift calendar alongside that common time basis. Because production still needs to assign events to the correct local working period. A UTC offset alone doesn't define a plant's calendar. The time zone supplies the rules needed to translate between UTC and local time, including daylight saving changes where they apply, while the shift calendar defines production boundaries. The repeated hour needs care when clocks move back. The same local clock reading can describe two different moments. So a local timestamp without an offset or other context can leave an event's position ambiguous. The spring change creates another trap. A shift crossing a daylight saving change can cover a different elapsed duration from its usual pattern, so calculate its actual boundaries rather than assuming every shift always spans the same duration. Precision time protocol, or PTP, supports tighter synchronization in suitable systems that often depends on compatible equipment and hardware time stamping. So specifying PTP in an architecture document doesn't establish that every machine can actually meet the required timing tolerance. Work with controls and network engineering on the design. Agree on approved time sources, permitted network paths, device support and behavior during loss of synchronization before changing clock settings on equipment connected to live production. Then measure the clock agreement you achieve for a fault investigation. Compare the observed offset and timing uncertainty against the separation between events, rather than judging success only by whether each device reports synchronization enabled carry synchronization health into the event record where practical, a timestamp from a device that has lost its reference may remain usable for some purposes, but the receiving process needs enough evidence to judge its timing. Don't silently rewrite questionable source timestamps, preserve the original timestamp, and if an approved correction is possible, retain the corrected time with the correction method and its uncertainty, so later investigations can distinguish evidence from adjustment that gives engineering an honest boundary for the diagnosis. Once the clocks agree within the required tolerance, there's still a separate issue to handle. Events can describe the correct time and reach the platform in the wrong order. Suppose the plant connection fails shortly before the night shift ends. The gateway continues storing events locally, and when the connection returns during the morning shift, it sends the waiting records into the platform alongside newly collected events. Those records still belong to the night shift. If the reporting process assigns them according to arrival time, it transfers production and interruptions into a working period where those events never occurred on the packaging line. The receiving process needs a rule for delayed evidence. It should assign an event to its reporting period using the agreed event timestamp, then assess whether that period still accepts automatic updates or requires a controlled correction that requires an explicit allowance for late arrivals for a shift total. Agree how long the calculation remains open after the shift ends based on normal delivery behavior and how soon production needs the result. Choose that allowance with the people using the number. A supervisor reviewing the handover might accept an early provisional total, while a business process relying on settled output may need a later version with fewer unresolved deliveries. Waiting longer has a cost. It delays a settled result while closing earlier increases the chance that valid events will arrive after publication and require someone to review their effect on an already used number. An event time window defines which period an event belongs to. The late arrival policy defines how long the processing system waits, and what it does when a record for that period arrives after the agreed waiting time expires. Keep those decisions separate in the implementation. A record can belong unambiguously to the night shift while arriving too late for that shifts routine update process, so lateness alone shouldn't move it into the morning. Total processing tools can enforce parts of that policy before relying on a streaming configuration. Verify whether events beyond its allowance get discarded, redirected, or handled another way because that behavior determines what evidence remains available for correction. The night shift total should stay provisional, while the gateway still holds undelivered records, even if the reporting process has successfully calculated everything that's reached it so far. You can't call that number final until you know the gateway is empty. Provisional needs a specific reason attached. Don't just slap a general warning on it that nobody can act on. State which source is still incomplete, what period that affects, and whether the total could change once delivery catches up. A deadline alone doesn't prove completeness. If the gateway still reports a backlog when the allowance expires, you can close the period under the agreed policy, but that unresolved delivery condition needs to stay attached to the published result closure controls revisions, but it doesn't prove the record is complete. Define who can approve a later change, which downstream users need notice, and whether the original published total must remain available beside the revised value for review. Now consider a stop event arriving after that closure. If it changes the calculated running time, the correction should identify the affected shift. The previous result, the revised result, and the delayed evidence that explains the difference. Nobody should discover that change by accident. A planner who acted on the earlier report needs an explicit correction notice when the revision affects the basis of that decision, with enough detail to reassess the action. Retain both event time and arrival time. Together, they let an investigator establish where the event belongs and when the platform could first have included it without pretending the reporting process knew something it hadn't yet received. The correction needs its own history to record when the revision occurred, which Late Record supported it, and whether approval followed the normal closure policy or an exception agreed for that reporting period preserved the earlier published version. Otherwise, someone reviewing the morning decision a week later could judge the planner against a corrected total that didn't exist when the planner released the next production order out of order. Delivery also affects calculations. Within an open period, a delayed stop may need to fit between previously received transitions, so affected intervals need recalculation using event order, rather than simply adding each arrival to the current machine state. That recalculation should leave a trace. Once delayed, records have settled into their proper periods. The next question concerns the measurement itself. A correctly placed pressure reading can still carry a scale that the receiving process interprets incorrectly. Let's give that pressure reading a specific unit problem. Suppose the packaging line reports pressure in bar, but the receiving calculation expects pascals and reads the incoming number without checking which unit the source actually supplies. Six bar and six pascals describe very different conditions. One bar equals one hundred thousand pascals, so interpreting the number six in the wrong unit changes the physical meaning before any pressure calculation even begins. Both values can pass a basic numeric check. The receiving process needs to compare the declared source unit with the unit required by the calculation. Then apply an approved conversion rather than trusting the field name. There's another layer below the engineering unit. A controller register might contain a whole number that represents tenths of a bar, which means the stored number needs scaling before it represents pressure. In bar, for example, sixty could represent six bar. Under that configuration, if the gateway already applies, that scaling a second division downstream would turn a correctly converted measurement into a value ten times smaller than the actual reading. I trace the conversion at every hand off. Ask what the instrument outputs, what the controller stores, what the gateway publishes, and which transformations the receiving process applies before anyone uses that pressure in a calculation. Write down the value at each step during an approved check. Compare those values against a known reference condition so engineering can confirm both the conversion formula and where that formula actually runs in the collection path. Some signals need an offset as well as a multiplier. A raw register representing the bottom of the instruments configured range might correspond to a non-zero pressure, so multiplying alone won't necessarily produce the correct engineering value. Use the configured range, not just the instruments advertised capability. An instrument capable of a broad pressure range can operate with a narrower configured output range, and the controller's conversion needs to match that actual setup. The pressure reference matters to gauge pressure measures relative to local atmospheric pressure, while absolute pressure uses a vacuum reference. So two records labeled bar can still describe different quantities that shouldn't be compared directly. That distinction belongs in the signal definition. If a calculation requires absolute pressure, engineering needs to approve how a gauge pressure source becomes suitable for that calculation, including where any required atmospheric reference comes from. For shared calculations, choose a canonical unit that's the agreed unit used after conversion, so downstream calculations receive the same physical quantity in the same form, regardless of how each machine originally reports it. Canonical doesn't mean every operator must speak in pascals. You can retain bar for an operating discussion while calculating in an agreed common unit, provided the conversion remains explicit and the reported label matches the value people receive. Keep the original value beside the converted result. Preserve the original unit and any raw register representation you actually receive. Because an investigator needs to distinguish a strange source reading from an error introduced during conversion. Tie the conversion rule to the signal's identity. Applying one formula to every field called pressure risks mixing different register scales, instrument ranges, and pressure references across equipment that happens to use similar tag names. Configuration changes need dated conversion versions. If maintenance replaces an instrument and changes its configured range, records collected before and after that change may require different formulas, even though the physical measurement point remains the same. Apply the version that governed the original observation. Delayed readings from before the replacement still need the earlier conversion, so using whichever formula happens to be current when processing runs can corrupt otherwise usable historical measurements. Check the converted result against known engineering limits. Those checks can expose a gross scale error, but a plausible pressure doesn't prove that the unit and conversion match the equipment, especially where different ranges overlap. An unknown conversion should remain unresolved if the source configuration can't establish whether sixty means sixty bar or six bar. Preserve the reading with that limitation rather than selecting whichever interpretation looks more comfortable. Once the scale is established, the pressure has a defensible physical meaning, but the delivery path can still send that perfectly valid reading more than once. Which raises a different question are we receiving another observation or another copy? A retry can look exactly like another completed pack on the packaging line. The receiving service might store a part complete event successfully, then lose its connection before the sender receives confirmation that the event reached its destination. The sender can't know whether storage succeeded. Reasoning the event protects against losing production records, but the receiver now needs a way to recognize an event it has already accepted rather than counting another pack. Repeated delivery doesn't necessarily indicate a broken connection design. It can follow normal recovery behavior. So I design production, counting around the possibility of retries rather than assume each delivered message describes a new physical occurrence. The event needs an identity that survives delivery. Assign that identity when the event is first captured, then preserve it through buffering and retries so every copy still identifies the same original occurrence at the source. A new identifier for every send defeats that purpose. If the gateway creates a fresh identity each time it retries, the receiver sees apparently distinct events. Even though the machine completed only one pack. Where equipment supports it, use a source event sequence combined with a stable source identity. That sequence can distinguish successive part complete events even when they carry the same value and occur within the resolution of the recorded timestamp. Restarts need their own context if the source starts its sequence again after a restart, include an identifier for that sequence lifetime so a new event doesn't collide with an older event bearing the same sequence number. Check where that restart identity originates. A gateway restart identifier won't explain a controller sequence reset unless the collection design reliably detects that reset and establishes a new event identity boundary before accepting subsequent records. Older equipment may provide less evidence if the source exposes neither a durable event identifier nor a reliable sequence. Document how the collector creates identity and what uncertainty remains before treating that method as proof. Timestamp matching alone can delete real production. Two distinct packs can complete within one timestamp interval, particularly where the source clock records time less precisely than the machine produces individual completion events during normal operation. Equal values aren't duplicate events either. Two pressure observations can carry exactly the same measurement while describing separate acquisition moments, and two part complete events can both contain the value one while representing different packs. Delete by event identity, not by resemblance. A rule that removes matching values within a short period risks removing legitimate events precisely when the line operates steadily and produces records that look much alike.

00:34:38 Speaker: I'd also compare copies sharing the same identity. If their payloads disagree, flag an identity conflict for investigation rather than quietly choosing one because a reused identifier or change payload could conceal a separate collection defect. Once identity holds, make processing idempotent. That means processing the same event again leaves the production result unchanged, while processing a genuinely new event still changes the result according to its meaning. Protect the stored event and its counting effect together. If the process records an event as handled before updating the total, a failure between those steps could cause a retry to skip production that never reached the total. Reversing those steps creates the opposite risk. Updating the total before recording completion can allow a retry to add the same pack again, so use an atomic processing design or derive totals from uniquely stored events. That behavior needs protection against simultaneous retries to two. Workers can both check that an event hasn't appeared yet, so a separate check followed by an unprotected insert doesn't establish reliable duplicate handling under concurrent processing. Remember the accepted identities for long enough. The retention period needs to cover supported outage, recovery and replay, because forgetting an identity before it's buffered copy returns allows an old event to count again. Let's talk about how to tell the difference between a legitimate repeated delivery and actual accepted production. When you track arrival records for those repeated events, you give your platform team the evidence they need to understand retry behavior without inflating your production totals. The key is keeping those repeated deliveries distinguishable from what you actually count as production. Now, some machines don't send individual completion events at all. Instead, they expose production through a rising cumulative count, and interpreting those changes requires a completely different set of rules than figuring out whether a delivered event already exists in your system. A cumulative counter records a running total, not another quantity you should add to anything. If your partner reports eight thousand completed packs and later reports eight thousand and twenty, the increase represents twenty packs. Assuming both readings belong to the same uninterrupted counter lifetime. Add those readings together and you'd invent production that never happened. Unlike individual part complete events, successive counter readings describe how far one running total has advanced. That means your calculation needs a valid starting point before it can establish any increase at all. The first reading establishes that starting point unless you have a trustworthy earlier reading for comparison, that first number doesn't tell you how much production occurred during the period you're reporting, even if the number itself looks perfectly reasonable. Here's where it gets practical you need to establish what causes that counter to reset in the first place. A machine restart might clear it. An operator might clear a shift counter manually. A recipe change might start a new batch count, while a separate lifetime counter keeps running in the background. A falling count needs explanation before you do any arithmetic. If the reading drops from eight thousand to twelve, subtracting the earlier value produces a negative difference. Replacing that difference with twelve assumes a reset you haven't actually established yet, and that assumption can lose production that happened before the reset. The machine might complete more packs after your last observation and then clear its counter, leaving those packs outside both the previous reading and the new count of twelve. A reset boundary needs evidence where the machine exposes a reset event or records a final count before clearing. Use that evidence to close the previous counter lifetime and establish the start of the next one. But here's the thing a restart doesn't always clear the counter. Some controllers retain counts through a power interruption, so treating every restart as a reset can introduce an error just as easily as ignoring a reset that actually occurred. Recipe changes need the same source specific check. Establish whether the counter continues, clears, or switches to another product specific register. A change in the displayed number might describe a different counter rather than progress within the original one. Rollover follows a different rule from an operator. Reset. A counter with a fixed numeric limit can return to its starting value after reaching that limit, and a valid difference must account for the defined wrap behavior. The limit has to come from the counter specification itself. Calculating across rollover also requires confidence that no more than the supported number of wraps occurred between observations, which becomes uncertain after a long collection outage. Ordering matters before you calculate any difference. A delayed reading inserted between two existing observations can change the valid neighboring comparisons, so process counter history in verified observation order within each known counter lifetime. Conflicting observations should interrupt that calculation entirely. If two readings claim the same observation position but contain different counts. Preserve the conflict rather than selecting whichever value produces a smoother production total for the shift. Then test each increase against physical capability. Compare the increase with elapsed time and an engineering approved production bound, allowing for the way the source updates its count, rather than assuming every read reflects an evenly spaced increment. A large increase isn't automatically a bad reading. Some machines expose a counter only after a batch finishes or update an external register in steps, so the plausibility check needs to reflect that documented behavior. Missing readings don't always destroy the total. If valid, readings surround a gap and the counter demonstrably continues without reset or rollover ambiguity. The difference can establish total production across that gap, though not its detailed timing. Shift boundaries expose that timing limit. If one reading precedes hand over and the next follows it, the increase spans both shifts. Assigning all of it to either shift would claim timing. The observations don't establish estimates. Need a separate label if production approves them. Otherwise, keep that increase as production known across the wider interval, with its allocation between shifts unresolved rather than distributing it according to an assumed steady rate for all katana. Publish only differences with defensible boundaries. Mark intervals affected by unexplained resets, implausible jumps, or uncertain ordering. Keep their unresolved contributions separate from confirmed production instead of silently dropping the interval. A defensible count still doesn't explain machine performance to judge whether the katana produced slowly. You also need to establish when it should have produced and which operating condition applied during those counted intervals. Production state needs an agreed definition before it becomes reporting data for the katana. I'd ask production and controls engineering to describe the conditions that distinguish producing, waiting, interrupted and unavailable using situations they can verify on the machine. Start with conditions people recognize during a shift, a cartoner waiting for incoming packs and a cartoner blocked by equipment downstream might both produce nothing, but they point the supervisor toward different places to investigate. Keep those distinctions beneath the broader stopped category that lets the shift report calculate non producing time consistently, while retaining enough detail for someone to distinguish an internal machine interruption from a line condition outside that machine. Then define how the available signals establish each state. The mapping should specify which combinations support a conclusion which signals take priority when several conditions occur together, and which combinations leave the state unresolved for reporting. Conflicting indications need an explicit presidents rule. If a fault indication and a waiting indication appear together, production and engineering need to agree which state applies based on the machine's behavior rather than the order messages happen to arrive. Precedents should resolve understood combinations, not conceal unexplained ones, where two indications contradict the agreed machine logic. The mapping should return unknown and identify the conflict instead of forcing that interval into whichever category appears first. Unknown belongs in the state model. It records an interval where the available evidence can't support an operating classification, so reports can account for that time without claiming either production or a confirmed stop. That interval still occupies part of the shift. If a performance calculation excludes unknown time, state that exclusion and its effect on coverage. Removing uncertain intervals can make the remaining operating period appear better than the full record supports. This becomes particularly sensitive with overall equipment effectiveness or OE. OE combines availability, performance and quality, and each factor depends on agreed definitions of the time, output and acceptable production included in its calculation. Availability compares operating time with planned production time production needs to define which scheduled periods belong in that denominator, including how breaks, planned maintenance and other agreed exclusions affect the time against which availability gets judged. Performance compares actual output with an agreed ideal production rate. That calculation needs a consistent treatment of interruptions, because assigning the same lost seconds to both availability and performance can charge the machine twice for one interruption. Microscopes expose that policy choice quite quickly. A plant might classify brief interruptions as performance losses and longer interruptions as availability losses, but the boundary and its handling need agreement before anyone compares results between shifts. There's no benefit in hiding that choice inside. Report logic. Document the duration threshold. How an interruption crossing it gets classified, and whether several short interruptions remain separate or form one sustained incident under the reporting policy. Keep that policy distinct from what the machine can observe. An interruption too short for the collection method to capture can't become a reliably measured micro stop, simply because the reporting team chooses a threshold that includes it. Quality also needs a defined counting basis for the OE calculation. Agree which output counts as total production and which output counts as acceptable production. So the quality factor describes an understood production boundary. Manual reason codes and explanation to recorded interruptions give operators a manageable set of reasons with clear meanings, and allow an unresolved reason when the cause hasn't yet been established during the shift or subsequent review, a reason code shouldn't silently replace the observed operating state. An operator may revise an interruptions explanation after investigation, while the recorded start and end of that interruption remains separate evidence about how long the machine stopped. Those revisions need a visible basis. Preserve who changed the reason when they changed it, and what supported the change, particularly when that explanation changes. How losses get assigned to departments or improvement work. Test the definitions against awkward cases before publishing the measure. Walkthrough changeovers, waiting periods, simultaneous faults, and incomplete indications with the people who work the line, then check whether the mapping produces classifications. They can explain even in agreed running state leaves another question open to interpret whether that running period represents expected production. The record also needs to establish what the machine was making and which production activity applied at that time. The same machine can behave differently across products on our packaging line. A heavier pack might require a different cartoning setup, so interpreting a running period means knowing which product and operation actually occupied the machine. That information usually lives beyond the controller. The manufacturing execution system, or Mez manages work execution and records production activity, which can provide evidence about the order and operation active when the machine produced an event. Check with the Mez record actually confirms an operation. Dispatch to the line, an operator signing into that operation and a recorded production start describe different stages so they shouldn't automatically establish the same active production context. The business record answers another part of the question. Enterprise resource planning, or ERP holds orders, materials and plans. But a planned assignment does not by itself confirm that the packaging line executed that assignment at the scheduled time. A schedule change makes the distinction practical. Suppose production swaps two orders during the shift because packaging material arrives late, while the ERP plan still shows the original sequence until someone updates the business record. Here's the problem most manufacturers don't talk about. You can attach perfectly plausible product details to the wrong production interval, even when neither the machine nor the order record contains an obvious formatting error. The original plan looks clean, but the context doesn't actually match what happened on the line. I'd connect context through three explicit relationships product, process, and resource. The product describes what we're making. The process identifies the operation being performed, and the resource identifies which machine or line carries out that operation. Under the recorded assignment, each one answers a specific operational question for a pressure observation. The process, step and product specification help establish which operating requirement applies, while the resource identifies the equipment and measurement point where that requirement needs interpretation. Keep those relationships separate from display names. If the order system calls an item a finished case and the execution system uses a packaging variant, the integration needs an approved mapping that explains how those records relate. A shared description isn't enough to prove equivalence. Two products can carry similar names while requiring different pack sizes or handling steps, so the mapping needs identities precise enough to distinguish the variants that affect production interpretation. Context also needs a time boundary record when an order, assignment or recipe becomes effective on the line and when it ends. So an observation can connect to the context that applied when it occurred. Use that effective interval when joining records. A context join means linking the machine observation to another record, and here the match depends on both the resource identity and the observation falling within the assignments effective period. Consider a recipe change during the afternoon. If an analyst links yesterday's pressure history to the recipe currently assigned to the katana. The calculation can assess yesterday's observations against requirements that didn't apply when those observations occurred. Preserve the earlier recipe assignment and its applicable specification version. The historical join should select those records using the observation time, rather than selecting whichever recipe happens to appear as current when someone runs the analysis. Changeovers need their own interpretation between the previous order ending and the next order starting. The machine may perform setup or trial production, so forcing that interval onto either order can misstate what the line actually produced. An absent or overlapping assignment needs investigation. If no context applies, retain the observation without an asserted order. And if two assignments claim the same resource and period flag the conflict rather than choosing whichever record arrives first. Some corrections reach execution records after production finishes, retain when the assignment applied and when its correction became known. So you can explain both the revised production history and the information available during the original shift review. Agree which records establish each part of the context production might confirm the executed operation through miss records. While the ERP record supplies the business order relationship with reconciliation needed wherever their identifiers are recorded, assignments disagree. Microsoft Fabric can bring those records into a shared processing environment. Your organization still has to define the relationships and timing rules, because placing machine history beside order tables doesn't establish which order actually occupied the line at a particular moment. Not every check needs that full context with the production relationships defined. We can decide which checks require those business records and which can run close to the machine before an observation leaves the plant. I'd place the first checks beside the collection process. Edge validation means checking observations on a local gateway or industrial computer, where the source connection remains available, and the work doesn't depend on reaching a cloud service. Keep that first pass small and predictable. Its job is to identify faults that local information can establish, then attach the findings before the observation moves beyond the plant and becomes harder to investigate for the partner. Begin with the structure of each collected record check that required fields exist, that their contents match the agreed types, and that the collector can associate the record with a configured measurement point. A type failure needs a clear result. If a numeric field contains text, retain that failure explicitly rather than converting the text to zero or letting an unchecked conversion stop the entire collection process. Required fields need different handling from optional ones. An absent diagnostic comment might leave the measurement usable, while an absent signal identity prevents the collector from establishing which configured rules apply to the value it received. Apply source status checks during that same local pass. The edge process should preserve the original status and record its assessment separately, so a local rule doesn't overwrite evidence supplied by the equipment interface. Engineering limits provide another useful local check for a measurement with a verified instrument range. Compare the received value against that configured range and flag values that the measurement chain shouldn't produce under its documented setup. Separate instrument limits from normal operating limits. A reading outside the preferred operating band could describe a genuine process problem, so the edge process should pass that observation with its warning, rather than dismissing it as faulty data. Local checks should explain their findings include the failed rule and its version with the observation, so someone reviewing an exception can establish what the gateway checked without guessing from a generic bad value label. There's a practical distinction between flagging and suppressing. A doubtful pressure reading may still help an engineer investigate the source, even if the local assessment says that it shouldn't support a current operating decision. Cloud disconnection shouldn't disable those checks. Where the gateway supports local buffering, it can continue recording observations and validation findings during an outage within the storage and processing limits approved for that installation. Store the findings with the buffered observation. Otherwise, a reconnect could send the value without the fault flag that applied when it was collected, leaving the receiving system with an incomplete account of the local assessment. The local process needs its own fault indication to if validation stops running or its configuration can't load, mark that condition explicitly rather than allowing unchecked observations to leave with the same status as checked ones. Keep collection and validation failures distinguishable. A rule engine crash doesn't establish that the instrument failed, and separating those conditions helps the plant team investigate the component that actually needs attention. Resource use needs an agreed ceiling check, processor load, memory use, and local storage under the expected collection rate, including the extra work created when many signals fail. Validation together during an equipment interruption. Avoid making each observation trigger expensive work load approved configuration locally and re-use it rather than asking the controller repeatedly for supporting details that could have been obtained through a less demanding collection arrangement. The deployment path matters as much as the rule controls and plant support teams need to approve where the validation software runs, how updates reach it, and who can change its configuration during production. Keep those permissions separate from Machine control. A service that reads observations and records quality findings doesn't need authority to alter machine set points, clear faults, or change the controller program to complete its reporting task.

00:52:29 Speaker: Test the failure behaviour before enabling the service, stop validation on the approved host and confirm that the machine continues its intended operation. While the reporting path exposes the loss of checking rather than concealing it. A stricter rule shouldn't create an equipment dependency. If local processing becomes overloaded, its documented response must protect machine operation and make the reporting limitation explicit, rather than consume resources needed by the production system. Now the observations can leave with their local findings attached. The next boundary is delivery accepted readings and flagged readings. Both need to travel without losing the distinction. The edge process has just established. Delivery needs its own evidence. A pressure reading can pass every local check and still fail to reach the reporting platform, so the transport path needs to report its condition independently of the measurements it carries. That path crosses an operating boundary. Operational technology, or OT, includes the system supporting physical production, while information technology, or it includes the enterprise services receiving and processing those records beyond the plant network. Connect those environments through approved network routes. Plant and security teams need to agree which component initiates each connection, which destinations it can reach, and what traffic can cross each boundary without opening unrelated access to production equipment. An outbound connection still needs controls, restrict its destination and permissions and confirm how certificates, firewall rules and any intermediate services will behave during maintenance or a prolonged interruption of the external connection. Microsoft provides transport services for different requirements. Azure IoT hub supports device oriented messaging and identity, while Azure Event Hubs handles high volume event ingestion with the choice depending on the architecture and how producers connect. Neither service establishes that a pressure reading is physically credible. Successful acceptance by a messaging service confirms a transport step, while the measurements existing quality findings remain separate, evidence that downstream processing still needs to receive. Be precise about what each confirmation means. A gateway recording that it sent a message isn't the same as a service. Confirming acceptance and service acceptance doesn't prove that the intended consumer has processed the message. Monitor those boundaries separately on our packaging line, the gateway could empty its outgoing queue while a receiving application falls behind, leaving production reporting delayed despite an apparently successful recovery at the plant. Queue age helps expose that distinction, track how long the oldest waiting record has been held because a modest queue containing old events can matter more to an operating decision than a larger queue draining promptly. Queue size needs a capacity reference. Reporting ten thousand waiting messages tells you little without knowing how much storage remains, how quickly records accumulate, and whether the connection can drain them faster than production creates them. Size the buffer around a supported outage use measured message sizes and production rates, allow room for bursts, and document how long the installation can retain records before its configured storage limit becomes a concern. Specify what happens when that limit approaches the collection service might stop accepting additional records or discard records under an approved policy, but either response needs an explicit loss indication and a notification someone can act on. You need to identify the affected source and estimate the time range. And here's the key distinction you need to separate a confirmed count of dropped messages from a period where you simply can't establish how many records went missing. Those are two different situations that demand different responses. When a connection fails, use whatever retry behavior the client supports, but build in increasing waits between attempts. The goal is to keep repeated tries from overwhelming the gateway or the receiving endpoint, while the underlying fault is still there. You don't want to turn a temporary glitch into a self-inflicted outage. Here's the real challenge a gateway that's sending stored events while still collecting current production data needs enough throughput to actually reduce its backlog. Your monitoring should tell you whether that backlog is shrinking, staying steady, or continuing to grow after reconnection. If it's not moving, you've got a capacity problem, not just a connectivity one, authentication rejection and unreachable endpoint, and a service refusing excess traffic all need completely different responses, so preserve enough diagnostic detail to route the issue to the right team without exposing credentials in the process. A generic connection failed alarm tells you nothing useful. Grant only the access that component's role requires. separate sending permissions from administrative permissions and keep credentials out of message payloads. Application logs and shared configuration files wherever possible. If a credential leaks through a log entry, you've created a security incident on top of whatever else went wrong. Assign someone specific responsibility for expiring certificates or secrets. Test the approved replacement procedure before you need it, and monitor authentication failures. So a routine security change doesn't quietly interrupt production reporting for an entire shift before anyone notices if your normal telemetry connection goes down, relying exclusively on that connection to announce its own failure leaves the receiving team completely unaware. So combine local indicators with monitoring that detects absent transport activity. The alert path needs to be independent of the thing it's monitoring. Report whether records are flowing, waiting or known to be lost. Include the affected source and name a specific response owner rather than just firing off a general connection alarm. Something's wrong somewhere doesn't help the person on call decide what to do. Once the receiving platform accepts the events, another validation stage begins. You need to use the source evidence and transport history that you preserved through that journey from the packaging line. Arrival isn't the end, it's the beginning of the next check. Before you reshape the carton as messages into convenient tables establish a landing stage. Keep the received content separate from whatever interpretation you'll apply during later processing. Don't let your transformation assumptions become irreversible. The route into Microsoft Fabric should match the source connection, the expected event volume, and the required arrival delay. Don't just grab whichever connector someone used successfully on a different project. Each integration has its own constraints. Check the source and destination support for your specific deployment, including message formats and limits, before assuming that the transport service you've selected connects directly to the storage you want. What works in one region or tenant might not work in yours. If the plant exports files periodically, a supported pipeline or file based ingestion route might meet the requirement without introducing a continuous processing service that your operating team doesn't actually need. Don't build a real time pipeline for data that only matters at the end of the shift. Include the larger payloads and bursts that occur during recovery. A route that handles ordinary traffic just fine, might behave completely differently when the plant sends several hours of buffered observations all at once. Test the edge cases, not just the happy path. It should mean the earliest retained representation of the received record, with its content preserved before any business transformations, not a table already stripped down to whatever fields a current report happens to use. Raw means untouched, not convenient. Alongside the extracted fields. Retaining the received representation lets an engineer check whether an unexpected result originated in the message itself, or appeared when processing interpreted its contents. That distinction saves hours of debugging. If the ingestion route converts a field, changes its representation, or can't retain part of the incoming message, document that behavior. Don't describe the stored result as an untouched original. If you've already transformed it on the way in record, which receiving endpoint accepted it when that acceptance occurred, and which ingestion, run or processing instance handled it. The stored event needs a traceable entry point into fabric, so you can reconstruct the journey later. Store ingestion metadata in separate fields. A platform generated status should never make it impossible to establish which assessment came from the machine's collection interface. Keep the source truth separate from your processing artifacts. If a later transformation produces an unexpected decimal value, the landing record should let engineering inspect exactly what fabric received without depending on the already transformed pressure column. The original is the reference point. A fabric lakehouse can hold retained event history in files and tables, making it a suitable option where downstream processing needs access to accumulated records across longer periods. Think about what you'll query and how often it supports event oriented analysis using Kusto databases, which works well for frequent queries across machine observations. Where investigation depends on filtering and comparing records over time. Choose the tool for the job. Choose around query patterns, retention needs, and processing demands. Use more than one destination only when the extra copy supports a defined workload that justifies its storage and operating cost. Every copy adds complexity. Your architecture needs to distinguish received records from records that have undergone checks, and both from the prepared data used by reporting and other consuming Applications. The platform gives you storage, not process design. The raw stage preserves incoming evidence. The checked stage contains interpreted records with recorded assessment results. The consumption ready stage organizes the accepted meaning around a defined reporting need. Each stage has one job. They can use separate tables or other controlled storage boundaries, provided each stage has a clear identity and downstream processes can't mistake an unassessed arrival for a prepared production record. Boundaries matter more than technology choices, right derived results elsewhere and retain a reference to their input records. That way, a failed processing change doesn't remove the material you need to rerun that processing correctly. Never let a transformation destroy its own source. Select a received message and confirm that its content source reference, arrival details, and original quality indications remain recoverable from storage through the access path. Engineering will actually use test the retrieval path before you depend on it. The next processing stage can examine those stored records against explicit rules, while keeping the distinction between what arrived and what the checks conclude about its suitability. That separation is the foundation of trustworthy validation. In fabric, I express each check so another engineer can run it against the same input and understand why it produced that result. If the answer depends on who ran it, the rule isn't defined well enough. Supported streaming transformations can handle suitable checks during ingestion. SQL queries or notebook logic can evaluate stored records. Pick the fabric, workload and processing approach that actually fits what each check needs to do. A required field check can evaluate one arrival, but detecting an unexplained counter decrease needs earlier observations, ordering rules, and enough retained history to establish which comparison actually applies. Some checks need context that streaming can't provide. A timestamp. Check needs a timestamp. If parsing fails, the timing assessment should report that it couldn't run. Rather than inventing a second finding about lateness, don't let a failed prerequisite produce misleading results. Distinguish a completed check from one blocked by missing input or unavailable reference data. Those conditions explain different limitations in what the processing could establish, and that distinction matters for troubleshooting. Instead of testing every message against one universal template. Select the expected structure for that signal inversion, then identify which required fields are absent or empty. Different signals have different contracts. If the contract requires a whole number count. Accepting a decimal and rounding it silently changes the source claim. Capture whether the received representation actually satisfies that requirement rather than silently coercing it. Test whether the asset identifier exists in the applicable asset mapping and whether the signal belongs to that asset. Don't accept any pair of individually recognized identifiers. They need to be a valid combination. Confirm that the declared quantity, source unit and applicable conversion configuration form a supported combination. Return a separate result when the processing can't establish that combination. Rather than guessing a rule. Can flag an observation dated beyond an approved future tolerance relative to recorded arrival, but preserve the distinction between a suspicious timestamp and a confirmed clock fault. Those need different responses in a notebook or suitable query workload, partition observations by their actual source and signal identity before ordering them. Don't let comparisons accidentally cross between different machines. A repeated identity with matching content differs from a repeated identity with conflicting content. The rule result should preserve that distinction for whoever investigates the collection behavior, generate the expected intervals from the applicable configuration and operating period, then compare them with usable observations. Don't treat every empty stretch of stored history as the same failure. A check can identify a sequence that skips a required intermediate state, but it should also record whether incomplete observations prevent a firm conclusion about what the machine actually did store, which preceding observation supported the calculation, which counter lifetime applied, and which engineering bound the increase exceeded. The finding needs to remain explainable without rerunning the entire history. Some findings need reassessment when additional records arrive, define which affected records get re-evaluated rather than assuming the first assessment remains correct throughout subsequent processing. Record the rule identifier version, execution time, input, record reference, and result include observed and expected details where those details explain the finding rather than merely adding storage, one describes when the machine record applies. The other establishes when this particular rule examined, it. That distinction matters when comparing results produced before and after a rule change. Here's the thing about changing a timing tolerance in engineering. You need to keep enough context to explain why the same input passed one version and failed another, but you also can't make it look like the machine's observation changed. The record itself has to carry that distinction before any receiving calculation uses the record. It needs to know which checks ran, which ones couldn't run, and which version of the rule supplied each conclusion. That's not optional metadata, it's how you establish whether the assessment is usable. The next step is deciding what each finding actually permits. You keep the record and its explanation available instead of making failed observations disappear. Without an account, a failure tells you something about the data. That information doesn't stop being useful just because the check didn't pass. Instead of leaving every consuming application to improvise its own response, define a handling policy upfront. That policy determines which users remain permitted and what has to happen before a restricted record can return to normal processing. Those categories describe permission to use the record for a defined purpose. They're not a permanent judgment that everything in the message is either trustworthy or worthless. A record can be perfectly valid for one use and completely wrong for another. A suspect record carries a limitation that permits only specified uses. A rejected record stays outside the intended calculation until someone resolves the reason for rejection. That's it. No moral judgment about the data, just a practical decision about what you can do with it. Take a pressure observation awaiting confirmation that reading might remain available for engineering investigation, but it gets excluded from calculations that require a confirmed measurement to assess whether operating requirements were met. Same data. Different permissions for different purposes. Keep the received payload, its source reference, the failed rule and assessment and every processing step that brought it into quarantine. Link all of that through a persistent case reference so someone can trace the full history without hunting through separate systems. Someone investigating the case needs to establish whether processing held the entire message, restricted a particular field, or blocked one intended use while allowing another to continue under stated conditions. That decision has to be part of the record, not something you reconstruct later. Put the policy into the controlled processing path. Consuming applications receive an explicit disposition rather than having to infer permission from a collection of diagnostic messages. The rule runs consistently regardless of who is looking at the output. If the payload can't be parsed, preserve the received representation and the parsing failure. Don't extract a convenient fragment and pretend the rest of the record passed its checks. The whole message failed, and the record needs to reflect that honestly. Handle them separately. So one unreadable payload doesn't repeatedly stop the consumer, but retain enough delivery information to identify its origin and investigate why that representation reached the platform in the first place. And otherwise, usable machine observation can wait for its assignment information. Once that information becomes available through the approved context correction process, run the affected checks again. The record isn't bad, it's just incomplete. After the agreed interval, keep the unresolved record restricted and raise an exception. Don't let repeated processing attempts consume resources indefinitely when the information available to resolve it hasn't changed. If engineering can't establish what the instrument actually measured. The correct outcome may be to retain the restriction and document that the observation can't support the intended calculation. Not every problem has a fix, and a proof correction needs supporting evidence and a stated method. Whether it fixes a malformed representation, restores an established mapping, or supplies context confirmed from the execution record, the correction has to be traceable and justified. Record who or what authorized it when it occurred, and which fields changed. Subsequent processing can use the correction without erasing what originally caused the restriction. The original failure stays visible, run the applicable checks again and record their new results. Don't move a record into accepted storage merely because someone marked the investigation complete. In a separate workflow, the checks have to confirm the correction works. The repair operation can have its own identifier, but creating another production event identity would let a corrected copy enter the count as though the machine produced an additional pack. That's how you end up with phantom inventory. If it did apply the approved replacement or adjustment path, rather than adding the corrected contribution beside the earlier one. Duplicate protection should cover retrieval of the repair itself. A case can finish with an accepted correction, continued restricted use, or a documented decision that the record remains unusable. That outcome should identify which downstream processing may proceed. Ambiguity at this point defeats the whole purpose. Track whether restricted records remain accessible and whether repair attempts complete. A table that quietly accumulates exceptions without a working release path only relocates the original problem. You need to know the system is working. The next test is whether the accepted records tell a consistent story when compared with other accounts of what actually moved through the line. A record can pass every individual check and still disagree with reality before comparing the cartoon account with the Mis total and the operators check. Establish whether all three describe the same material crossing the same production boundary during the same period. If they're measuring different things, the numbers won't match and shouldn't. Suppose the cartoon accounts sealed cases before inspection, while the mess records cases released after inspection and the operator checks cases physically waiting for transfer into finished stock, those totals measure different stages of the same process.

01:09:54 Speaker: No missing production required, provided those intermediate quantities can be established rather than assumed. To balance the numbers, you need actual evidence, not arithmetic. Convenience. For our packaging line, that might mean reconciling cases, leaving the carton against their subsequent disposition, accepted cases rejected cases and cases still awaiting a recorded decision all need to be accounted for. A rejected pack removed before case formation won't explain a difference between two case counts in the same way as a complete case rejected after the carton has count point, the timing and location of removal matters. Then establish whether the source total includes that material and whether the corresponding scrap record uses the same quantity basis. This gets tricky where operators record loose packs, but the execution record reports cases. A case that passes the count point, returns for correction and passes again can produce two valid machine detections while contributing only one accepted case to the completed order. That's a counting boundary issue, not necessarily duplicate delivery. Don't subtract repeated detections merely because the final production total seems too high. The machine detected what it detected. The question is whether the process design accounts for that behavior. An operator may complete a physical check before the mess receives the corresponding production confirmation. Both accounts are defensible at their respective observation and recording times during the handover. The disagreement is about timing, not accuracy. If one source remains open for additional postings, identify that condition and compare again at an agreed point. Don't treat every early difference as a confirmed discrepancy requiring correction. Establish what was counted, whether partial pallets were included and whether material moved during the check. A physical count also needs a clear scope before it can resolve disagreement. The operator isn't automatically right just because they're standing on the floor. The machine may provide the strongest evidence of passages through a particular detector. The Ms. establishes recorded acceptance against an operation. Neither account automatically answers the other's question for release production. The approved acceptance record might govern for a physical stock investigation, you might need a controlled count of material in a defined location before anyone adjusts the business record. Different purposes. Different authoritative sources. Naming an authoritative source establishes the normal basis for a decision, but contradictory physical evidence should still trigger investigation rather than disappear because a designated system reports a different quantity. Authority isn't infallibility include the source totals, common boundary comparison period, known differences, and remaining unexplained balance. The investigator needs to distinguish an understood production movement from an unresolved mismatch. A difference associated with rejected material might require production review. A mismatch concentrated at one detector might need controls engineering to inspect its behavior against actual case movement. Different problems need different expertise. Under an approved observation procedure, someone can compare actual passages with recorded detections. They check whether closely spaced cases, damaged packaging or handling practices affect what the detector registers during production. You can't always solve this from a desk. A short physical check can establish a particular fault under tested conditions, but it doesn't justify changing every historical total unless evidence supports the affected period and the size of the correction. Don't extrapolate from a single observation. An unexplained difference should remain an exception with the stated investigation outcome, even if that means the shift retains a quantity that production can't yet defend as confirmed output. Forcing the numbers to match just hides the problem. We can now distinguish records that satisfy their individual checks from records that also agree across defined production boundaries. That gives the next quality measures a much firmer basis. If someone describes the carton of feed as ninety eight percent complete, I'd want to know exactly what they expected to receive and which period that percentage covers. A percentage without a defined denominator is just a number. Completeness measures whether the expected evidence is present. Freshness measures whether it meets the age requirement. Validity measures. Compliance with defined checks. Consistency measures agreement across specified comparisons. Each one tells you something different. For completeness, define the expected observations or cover duration. Then state how much of that expectation the received evidence satisfies under the applicable collection agreement. For that signal, the basis has to match what you're actually measuring. A signal's expected behavior might require coverage across operating intervals rather than a particular message count. The measure needs to reflect the obligation you are actually testing, not just what's convenient to count. You could measure the proportion of required decision points where an observation met its age limit, instead of only judging freshness when a new message happened to arrive that prevents a misleading result during an outage. If you evaluate only received messages, a source that stops delivering can disappear from the calculation precisely when its absence matters most to the people waiting for current information. Report how much eligible data passed the required checks, but keep failed assessments and assessments that couldn't run distinguishable. Don't treat unavailable assessments as successful validation, because that hides the real gap. State which completed comparisons, met the agreed tolerance and keep periods awaiting a counterpart separate. That way, an unperformed comparison can't improve the apparent agreement between production records. A month of mostly healthy observations can conceal a serious failure during one shift. Choose windows that preserve the operating periods where decisions actually depend on the signal for the packaging line, I'd retain shift level quality measures. A longer trend can help establish whether reliability is improving, but it shouldn't replace the result for the specific shift whose production evidence Someone needs to assess if a required identity check fails. Other successful checks shouldn't compensate for it. The processing still can't establish which measurement the record represents for the intended use. That's a hard failure. Gate. Define which failed requirements block a particular use, then apply that permission separately from any score. That summarizes how well the remaining requirements performed over the reporting window. If engineering uses one document, why each component receives its weight and test whether a serious failure could remain concealed beneath strong results from less consequential checks, weights express a choice, not a physical property. A combined score should support that stated choice ranking recurring collection problems, for example, rather than becoming universal permission to use a signal for every production decision, keep the component results accessible. Beneath that score, someone reviewing a decline needs to establish whether observations disappeared, arrived too old, failed a measurement check, or disagreed with another account. Without reverse engineering the arithmetic behind the grade, a high volume source can dominate an event weighted plant score, leaving a less talkative but decision critical machine with poor data almost invisible inside an otherwise healthy total. Retain quality by asset signal and intended use that allows the same. Observation history to receive different suitability assessments where the requirements differ without suggesting that the underlying measurement itself changes according to who asks for it. Keep the calculation version, reporting window denominator and applicable requirements. That way, a later review can distinguish an improvement in source behavior from a change in how quality gets counted. A percentage should never conceal how little was tested, include the assessed population and unresolved portion, because excellent results from a small evaluated subset don't establish that the rest of the shift meets the same requirements. The next step is to carry that strength into the reported production number, so its apparent precision doesn't exceed what the machine records can support. Power BI needs to publish the number and its qualification together. Otherwise, a carefully assessed production record can reach a report that strips away its limitations and presents a stronger claim than the evidence allows. I'd established that behavior in the semantic model, the shared layer of relationships and calculation definitions that lets different reports use the same meaning for confirmed production. Estimated production and the periods those quantities describe. Give those quantities separate measures with explicit names. A confirmed output measure should include only contributions permitted under the agreed policy, while an estimated output measure keeps approved estimates identifiable instead of blending them into a single unexplained total. That distinction should survive ordinary report use when someone selects the night shift or a particular order. The associated qualification needs to follow that selection through the model's relationships, rather than remain attached to an unrelated plant wide assessment. Consider a supervisor asking for the Katona's completed output. The response might include a confirmed quantity and a separately stated estimated contribution, with enough explanation to establish whether their combination is suitable for the supervisor's immediate decision. Avoid calling the confirmed portion the complete shift total. Where part of the shift remains unresolved. That label would imply a reporting boundary the evidence hasn't covered, even if every included contribution qualifies as confirmed production. If the reporting layer contains only received observations. A period without observations may contribute nothing to the calculation, and leave no indication that part of the expected period remains unaccounted for. Use the reporting calendar to retain those periods. Then define whether the measure returns an unknown result. A partial result or a permitted estimate, rather than relying on whatever a particular report happens to display for an empty calculation. Zero should remain an explicit production claim. Configure measures so that an interval with no defensible production evidence doesn't become zero, merely because someone wants every reporting period to contain a convenient numeric result for comparison. A provisional period needs a machine readable status that reports can apply consistently, so the number remains identified as provisional. When someone changes the reporting period or groups several shifts together. Mixed periods need a defined result. Two. If a weekly quantity includes settled shifts and one provisional shift, its overall status should disclose that unsettled contribution rather than inherit the status of whichever shift the calculation encounters first. Coverage must match the measure being qualified, the output quantity, its usable coverage and its unresolved duration need the same asset and period boundaries. Otherwise, the report can pair a restricted machine total with reassuring coverage calculated across unrelated equipment. Keep the explanation close to the operational measure. Someone receiving the number through an exported report or a spoken briefing should also receive its qualification, without needing access to another report page to discover that the result remains incomplete. One describes when power BI refreshed its imported data or queried its source. The other describes when the underlying machine observation occurred and whether that observation meets the decisions age requirement. A recent refresh doesn't establish recent machine information. If collection has stopped upstream, power BI can successfully refresh against unchanged stored records, so reporting only the refresh time gives users confidence about the wrong part of the path. For current condition measures, publish the latest usable observation time, calculate its age using a verified time basis, and define how the report changes its wording or withholds a current state claim. Once that age exceeds the permitted limit. Check how that age calculation itself updates in an imported model. A stored age can remain unchanged between refreshes, so its calculation and report behavior need to match how frequently users expect the freshness assessment to advance. Ask whether production can quote confirmed output, identify the unresolved portion and distinguish a current machine condition from the last condition recorded before collection stopped supplying usable observations. Those answers should come from shared measure definitions with the qualification carried through consistently, the next question becomes more specific which source observations, processing decisions and reporting boundaries account for the particular quantity being discussed in the production meeting. I'd want that question answered without rebuilding the calculation by hand. A published shift quantity needs a traceable connection to its inputs, so someone can investigate the number after the original processing run has finished. That connection is data lineage. It describes the path from source observations through the conversions, checks, context assignments, and calculations that produced a particular result, including where processing changed or restricted the contribution of an input. Give each safe shift result a persistent reference associated with the resource reporting period, calculation version and processing run that produced the quantity people received in the meeting. A shift label alone won't identify the result precisely enough. If the night shift calculation runs again after a correction, the saved reference should distinguish the earlier publication from the later publication without relying on someone's memory. Then connect that result to its input population for the partner record, which observation periods and source records contributed, together with the boundaries used to assign their contribution to the reported shift quantity. You don't need to copy every input into the result record. A controlled relationship to an identifiable input set can provide the connection, provided that set remains recoverable and doesn't change underneath an already published calculation. Capture the processing versions that interpreted those inputs. The lineage should identify the applicable conversion mappings, validation assessments, production context versions, and measure definition, rather than merely naming the application that happened to execute the work. A software name can't explain a production decision. Knowing that a notebook processed the count tells you less than knowing which version ran, which inputs it read, and which configured rules governed its output at that time. Fabric can provide lineage information between supported platform items that helps establish dependencies, but explaining an individual shift quantity generally requires additional records connecting the published calculation to the particular observations and decisions that produced it. Build those connections into processing rather than reconstructing them afterward. When a transformation writes its output, it should also preserve the references needed to identify its inputs and configuration within the detail required for the intended investigation. Now consider an excluded contribution. The investigator should be able to establish which interval or event was left out, which assessment justified that decision, and how that exclusion affected the published quantity or its coverage. Here's the problem most manufacturers don't talk about an exclusion count alone leaves too much unanswered. Production needs to know whether those exclusions cover a brief period right after handover, or an interruption in the middle of the disputed order, because those two situations support completely different interpretations. An estimate needs the same level of clarity. You have to link its contribution to the affected interval, the estimation method and version, and the supporting observations, so a reviewer can tell the difference between the approved calculation and the evidence used to back it up. Make that account understandable without forcing someone to dig through processing logs. A shift leader should be able to ask why a contribution is still estimated, and get a plain explanation that names the missing evidence and the method used. Correction history belongs in the same chain. Starting from a revised quantity, the reviewer should be able to reach the earlier publication, identify what changed in the inputs or interpretation, and establish which specific correction accounts for the difference. Retention determines how long those explanations stay possible. You need to agree how far back production, maintenance and audit reviews need to investigate a result. Then keep the supporting inputs and processing definitions for that stated period. Different evidence can have different retention periods. Detailed payloads might expire before aggregated results, but the retained result should disclose when its underlying evidence is no longer available. Rather than implying that full reconstruction is still possible, access needs the same deliberate treatment. Give authorized investigators a supported route to the evidence, while restricting sensitive production details and recording access where required, without granting permission to edit the records under investigation. Test the route with a real published shift result. Ask someone outside the pipeline development team to trace one disputed contribution and explain its treatment using the retained records, not private notes from the person who built it. That exercise exposes gaps that documentation can miss. Once the quantity has an explanation that production and engineering can follow, the evidence can support a more useful question which source condition needs attention so the next shift doesn't inherit the same uncertainty? Maintenance needs a symptom. It can investigate, not a failed rule notification. I'd turn the evidence into a service request that describes the affected measurement, the observed behavior, and the production consequence. Without claiming a diagnosis the records can't establish for our partner. That request might concern intermittent pressure acquisition. It should identify the physical measurement point and explain when acquisition failed, whether communication recovered without intervention and which operating periods lost usable pressure evidence as a result. Describe what still worked during the affected period. If other measurements from the same controller kept arriving, include that observation because it helps the technician narrow the investigation without assuming the entire collection connection failed. A frozen value report needs a different description state, which expected process response didn't appear in the measurement, which confirmed operating changes, supported that expectation and whether the finding occurred once or repeatedly under comparable conditions. Clock drift needs evidence of its progression, provide the observed clock offsets and affected device identity, while distinguishing measurements against an approved reference from apparent timing differences inferred only by comparing unrelated machine events across systems. Calibration concerns need careful wording as well. A persistent disagreement with a suitable reference can justify checking an instrument, but a data rule alone shouldn't declare that the sensor requires adjustment before engineering examines the measurement conditions. Group related findings before creating maintenance work. If one acquisition interruption produces hundreds of failed assessments create a coherent issue covering that episode. Rather than sending a separate request for every observation the collector couldn't obtain. Grouping still needs boundaries. Someone can explain combine findings by the affected equipment and likely shared symptom, while keeping separate episodes distinguishable so the technician can establish whether the problem persists or returns after apparent recovery. Attach evidence the technician can use at the machine. Include the asset reference, measurement point, affected periods, relevant diagnostics, and a short account of the observed behavior with links to further records where deeper investigation becomes necessary. Access shouldn't depend on knowing the pipeline developer. The person assigned to investigate needs a supported way to retrieve the evidence, including enough local equipment detail to locate the affected channel rather than just its cloud identifier. Route the request according to the suspected failure location, an instrument or input channel symptom belongs with the appropriate maintenance or controls team, while a gateway acquisition defect needs someone able to examine the collector and its connection behavior.

01:27:16 Speaker: Processing defects need a separate route if retained, source records remain usable, but a transformation interprets them incorrectly. Send that investigation to platform support. Rather than asking a technician to inspect healthy equipment for a cloud side error, some cases won't reveal that boundary immediately. Start with the evidence available, record the investigation performed and transfer the case with its findings when the fault points elsewhere, rather than opening disconnected requests that lose the history. Work completion needs an observation based check, a replace connector or corrected device setting records an intervention, but the data issue should remain unresolved until subsequent acquisition demonstrates the behavior the repair intended to restore. Choose that check around the reported symptom for an intermittent failure. One successful read proves little, so agree on an observation period and operating conditions that give the suspected fault a reasonable opportunity to recur. Check the response without creating unsafe production demands. Wear confirming the repair requires a machine transition or reference measurement. Use an approved procedure and coordinate the work with the people responsible for operating the equipment. Record the outcome even when no fault appears. Note what the technician inspected, which conditions the check covered, and whether the original symptom remained absent, remained present or couldn't be reproduced during the investigation period. An adjustment should not rewrite earlier uncertainty by assumption. If an instrument passes its post work check that establishes evidence about its condition, then, while any conclusion about the affected historical measurements still needs its own engineering basis, the closure record should reach the reporting support process, connect the maintenance outcome to the original data issue so its status reflects verified recovery, continued restriction, or further investigation, rather than merely the completion of a work task. That connection needs someone to keep it moving when a technician finishes the physical repair, but the reporting restriction remains, a named person must confirm the checks and carry the outcome across the team boundary. Someone needs responsibility for the issue through to resolution. Assign one coordinating owner who keeps the case moving even when the investigation requires several teams and the eventual repair belongs to a different specialist, that role needs authority to request a response. Without an agreed escalation route, the coordinator can spend days forwarding messages between departments while the same disputed quantity continues. Reaching the production meeting each morning. Separate that coordination role from ownership of the underlying definitions. Production should own the operating meaning of a reported condition, including whether it supports the decisions supervisors and planners actually need to take during the shift. Controls engineering owns a different part of the agreement. That team establishes how the equipment exposes the condition, and maintains the connection between the approved operating definition and the controller logic or interface supplying it. Neither team should decide the whole question alone. Production can't establish what a controller bit does by naming it while controls engineering needs production input before turning that bit into a business definition of productive time. Platform ownership begins with the supported collection and processing path. Name the team responsible for keeping that path working, maintaining its configuration, and investigating whether an interruption occurred in acquisition, delivery, storage or transformation. Analytics needs an equally explicit responsibility. The owner of a published measure should control its calculation definition and ensure that reporting changes preserve the agreed treatment of restricted data, rather than quietly changing the production claim. Those responsibilities need named contacts, not just department labels. Record a primary owner and a covering role for each supported area, so an unresolved issue doesn't wait for one particular engineer to return from leave. Decide who takes the first action when evidence remains ambiguous for a disputed shift quantity, the reporting support owner might coordinate the investigation while production decides whether the available result can still support the immediate operating decision. Escalation should follow consequence as well as elapsed time. A fault affecting an imminent order commitment deserves a different response from an unresolved historical detail. Even if both cases entered the support queue together earlier that morning. Agree those response categories before an incident state who must acknowledge the issue, who can restrict the affected measure, and who needs notification when the investigation can't establish a defensible result within the required decision period. A hand off needs acceptance from the receiving team. Keep the current owner responsible for coordination until that acceptance occurs, so forwarding an email doesn't count as transferring responsibility for an issue nobody else has agreed to investigate. Recurring faults also need ownership beyond individual case closure. If the same condition returns after repeated interventions, assign someone to review the recurring pattern and decide whether the source, design or operating procedure needs a permanent change. Changes deserve the same discipline as incidents. A tag edit, firmware update, or recipe revision can alter what reporting receives, so its approval process needs to include the owners of the affected downstream interpretations. Use an impact check before scheduling the change. Ask which signals might behave differently, which consumers depend on them, and whether the existing contract mappings or published measures still describe the proposed configuration accurately enough. the change requester should identify those dependencies. Receiving teams then confirm their own required actions, rather than leaving the controls engineer to guess which fabric process or power BI measure might depend on the altered source. A mapping edit needs an accountable approver to someone. Correcting an asset relationship can redirect many observations into another machine's history, even though the equipment continues operating normally and no controller program changes at all. Validation needs controlled approval for the same reason you don't just flip switches in production when you relax, a rule you might let in things you previously blocked. When you tighten one, you might lose confirmed coverage you were counting on. Either way, the person who owns that measure needs to understand what the intended reporting consequence actually is. A planned maintenance window isn't good enough if the implementation slips. Processing needs the real effective date so it can interpret data collected around that change correctly. If you don't know exactly when the new rule started applying, you'll assign observations to the wrong configuration and your reports will be wrong before you even start reading them. When something breaks at two in the morning, plant teams shouldn't have to choose between restoring production and leaving an undocumented mess behind. Allow an urgent procedure with a named decision maker and a promise to document it properly afterward that keeps the plant running without creating a black hole in your change management. Once you've sorted out who owns what and who can approve changes, the real question is whether the revised path actually behaves the way those owners expect. You need to prove that before it touches production. Take an isolated environment with realistic cat owner messages. Change the inputs without worrying about test data leaking into real production reports or writing anything back to the equipment. That's your sandbox. Build a small event history where you already know the production count and the operating intervals. Write down what the output should look like before you run anything through the pipeline. That's your baseline. Have someone else check the test data independently. Two calculations that share the same faulty assumption can agree perfectly and still approve garbage. You need a second pair of eyes that doesn't share the same blind spot. Delete specific events from your test history. Verify that the system spots the missing coverage and handles the affected calculation according to the rules you agreed on. Make sure it doesn't accidentally mess up unrelated intervals that still have good data. Move the clock forward. Move it backward. Introduce a gradual drift each time. Check that the system identifies the affected records without inventing changes that never happened on the physical line. The test should prove whether the system picks the right configuration for the new unit, or rejects a combination it doesn't support. Don't let it pass a reading just because the number happens to fall inside the normal range. Scent matching records together send them through different processing threads. Resent them after the first batch finishes. Every time confirm that the final production count stays the same. Duplicate handling isn't proven until you've tried every way duplicates can actually arrive. Include a documented counter reset and a drop that nobody can explain. Specify the expected outcome for each one, then verify that the system doesn't invent an increase where none happened and doesn't lose valid production outside the affected window. While your test rig is running, pull the plug on the connection between the source and the receiver. Verify what stays buffered on the sending side and what the receiving side reports while it's sitting there with nothing coming in. When the network comes back, the backlog and the new data will arrive together probably in a different order than they left. Compare the final input set and the calculated output against the known history. If the numbers don't match, your recovery, logic has a problem. A final total can balance out even when individual transactions end up in the wrong time periods. Inspect the interval assignments and the intermediate results before you declare the recovery test a success. Feed the same retained input through the pipeline. Again, the output should be identical unless you deliberately changed a rule in between. If something changes, it should follow the expected correction path not appear as brand new production. Take your candidate algorithm and run it against a year's worth of real production data. Compare what it finds with what the current system reports. Don't change the published numbers until you understand exactly what would be different. A rule that catches every known fault might also flag every routine change over or normal variation. You need to look at the false positives alongside the real catches. If engineering considers those events acceptable, your rule is too aggressive. Don't just tighten the threshold until that one example happens to fail. Figure out whether the rule lacks the right evidence, uses the wrong time window, or evaluates the wrong thing entirely. The explanation matters more than the quick fix. Every time you find and fix a real problem, add its test case to the suite from now on. Any change to the system has to prove it. Didn't reintroduce that exact bug. That's how you keep from fixing the same thing twice. Don't just test the code. Verify that the configuration files, the reference mappings, the permissions, and the destination settings all match what the receiving environment expects. A perfect algorithm pointed at the wrong configuration file is worse than useless. Restore the old version against your test data. Figure out how you'll handle the records that the new version already processed. Write down exactly what needs to be replayed and what needs to be replaced. Don't figure this out. During a crisis. Write down which records need to be corrected or replaced. Then verify that your recovery process preserves the audit trail and doesn't accidentally double count anything while you're putting things back the way they were. Disable a critical lookup service. Stop the validation worker. Then verify that your monitoring system reports that checks are failing. Not that everything is quiet and peaceful. A silent failure is the most dangerous kind. Once you've demonstrated the expected behavior and practiced the recovery procedures, you can choose a small, safe starting point for the real rollout. Pick something where you can investigate every unexpected result without overwhelming the team. On our packaging line, I'd start with the planner trying to figure out how much work is left on the current order. That handover question causes arguments every shift. If we can make that decision cleaner, everything else gets easier. Start with the final count point the signals that interpret that count and the order assignment logic. Leave the pressure sensors and the vibration monitors for later. Adding them now would create more questions than answers without helping the specific decision we're trying to support. Those faults deserve attention, but they're not helping us decide whether the current order is finished. Keep them outside the pilot boundary and focus on the one question that matters right now. Can the production team explain the reported count? Can they identify what's still unresolved? Can they tell the planner whether that uncertainty might change the amount of work they need to schedule next. If the answer to any of those is no. The pilot isn't ready. Follow the production number from shift handoff into the planner's spreadsheet. Include every phone call, every spreadsheet correction, every gut check adjustment that happens before anyone actually acts on the number. If you don't understand the current process, you can't improve it. If the shift supervisor routinely calls the operator and adjusts the reported count by hand, that's not cheating. That's how the process actually works. Document that intervention and treat it as part of the baseline, not as an exception. You can ignore which shifts require manual investigation. How long does the disagreement stay open? Does the planner wait for an answer or just pick a number and move on? Document all of it. The pilot needs to prove it can reduce that uncertainty, not just move it somewhere else. The baseline exists to show what the current process demands from people. The pilot needs to prove it can reduce that burden, not prove that the old way was wrong. Keep the focus on improvement, not blame. Don't let the pilot control anything yet. Just compare its output with the numbers the plant actually uses. Run them side by side under the same conditions and at the same decision points. Let the data speak before you trust the new system. When the numbers match, sample the reasoning anyway. Matching totals can hide different interpretations of the same underlying events when they disagree. Dig into the evidence and figure out which interpretation is right. Don't hand someone a spreadsheet with a thousand rows and ask them to certify the whole thing. Bring them the one specific interval or event that happened during their shift. Ask them a question they can answer from memory. What happened on line three between two and three a m? Record. Whether the operator confirms the system's interpretation provides context. The system missed or leaves the question unresolved. Use that feedback to improve the next iteration. If operators keep disagreeing with the system, the system is wrong. Production and the planner need to agree on what evidence is good enough. What level of uncertainty is acceptable, and how exceptions get handled when they appear outside business hours. Don't negotiate these terms after you've already invested months in the project. Pick one published result without telling the engineer which one you'll choose. Then ask the support team to trace that number back to the raw data and explain exactly what contributed to it and what assumptions were made along the way. If they can't do that in five minutes, the traceability isn't good enough. How much unresolved uncertainty can the planner tolerate? What happens when that uncertainty could change the remaining work enough to alter the next scheduling decision? Answer those questions before you turn the system loose. An exception raised at three in the morning needs to reach the person on call, get acknowledged, and get resolved without waiting for the engineer who built the system to wake up. If the night shift can't handle it, the system isn't ready for production. A technically perfect system that requires the shift supervisor to spend an extra hour every shift. Investigating edge cases is a failure. The people doing the extra work get a vote on whether the improvement is worth the cost. If exceptions keep coming back unresolved. Hold the boundary where it is. Adding another production line would multiply the unresolved work, not build a supported process. Fix the feedback loop first, then expand that production number that supports the shift handoff will eventually feed a scheduling system, a digital twin, or an AI application. In those systems, the qualification that made the number trustworthy can disappear completely. A person might pause when they see a weird number. An automated system will accept it and propagate the error across three orders. Before anyone questions the original input, the qualification has to travel with the data, not stay locked in the spreadsheet. The planner now gets a completed count, with the unresolved pieces clearly separated. When a shift report changes, they can see exactly how that revision affects the remaining work. That's the foundation we've been building toward passing. Just the final number strips away everything the pilot needed to build trust in that number. The next system needs the same context, not just the answer. Think of it as a maintained digital representation of the production resource, its relevant relationships, and the operating conditions supported by available observations. A single number in a database field doesn't capture any of that. If you're building a digital twin, build it right from the start. The twin should only report what the evidence actually supports right now. If the katana's last verified condition happened before a data gap, the twin needs to remember that limitation. Instead of pretending the machine is ready to run, an explicit unknown state gives the system something real to work with. A planning engine can flag it for review. Hold that work order or run alternatives under different assumptions. Instead of treating an unconfirmed status as a confirmed machine condition, those responses need your organization's rules behind them. Microsoft tools can store the data and run the logic, but your team still decides what an uncertain status actually means and which decisions need a human check before moving forward. Scheduling adds another layer. Here, the output is only as good as what you feed it. Advanced planning and scheduling apps uses resource constraints and production data to build schedules, but a plan that looks solid on paper might depend on a machine condition nobody has actually verified. Take that Cartoner assignment. If its availability is still uncertain, the scheduler can calculate two versions one with the machine, one without. Then flag which decisions hinge on confirmation instead of publishing one answer that might be wrong. That turns uncertainty into something the planner can actually use. They can see which schedule is backed by confirmed data and which one only works if production answers a specific question about that line first. Optimization still needs more than clean sensor data. Resource capabilities. Material availability and process rules have to come from maintained production models. Better sensor data helps scheduling without replacing the entire model. That scheduling actually depends on. Industrial AI brings a different problem if you train a model on production history that includes estimated values and later corrections, the model learns those patterns. Unless your training pipeline separates real observations from estimates and fixes, the training data needs the quality tag alongside the value itself. Depending on the task, you might exclude uncertain periods and missingness indicators, or test how the model behaves. When you remove estimated data points from the training set, that choice has to match what the model will actually face. If you're building a predictor for remaining useful life, test it under the same data gaps it will hit in production, not just on the clean historical windows where every sensor arrived on time. Generative AI needs qualified retrieval, too. When an assistant pulls a shift total from fabric, the answer should include when that data applies and what its limits are. Otherwise, a partial number turns into an unqualified claim about what was actually produced. The recommendation needs a boundary around what it actually knows, and the assistant can explain scheduling trade offs, but claiming the cardinal can take another order. Need support from the actual resource state and constraints, not just a fluent guess based on yesterday's output. I'd want every AI response to flag its own open dependencies. That way, the planner asks for confirmation before acting, while the deterministic engine still tests the final schedule against the manufacturing rules the organization has actually approved, the unknown status has to survive the whole pipeline. If the Twin Records uncertainty, but the planning tool converts it to available or the AI drops it from the answer, the architecture just recreated the same false certainty it was supposed to fix. Every production number needs evidence or a clearly stated limit. When neither exists, that number stays outside the confirmed total, even if including it would make the morning Report look better. Which data gap causes the most headaches in your reporting right now? I'd like to hear how you're handling it.