LLVM  8.0.1
Public Member Functions | Protected Member Functions | List of all members
llvm::RegPressureTracker Class Reference

Track the current register pressure at some position in the instruction stream, and remember the high water mark within the region traversed. More...

#include "llvm/CodeGen/RegisterPressure.h"

Public Member Functions

 RegPressureTracker (IntervalPressure &rp)
 
 RegPressureTracker (RegionPressure &rp)
 
void reset ()
 
void init (const MachineFunction *mf, const RegisterClassInfo *rci, const LiveIntervals *lis, const MachineBasicBlock *mbb, MachineBasicBlock::const_iterator pos, bool TrackLaneMasks, bool TrackUntiedDefs)
 Setup the RegPressureTracker. More...
 
void addLiveRegs (ArrayRef< RegisterMaskPair > Regs)
 Force liveness of virtual registers or physical register units. More...
 
MachineBasicBlock::const_iterator getPos () const
 Get the MI position corresponding to this register pressure. More...
 
void setPos (MachineBasicBlock::const_iterator Pos)
 
void recede (SmallVectorImpl< RegisterMaskPair > *LiveUses=nullptr)
 Recede across the previous instruction. More...
 
void recede (const RegisterOperands &RegOpers, SmallVectorImpl< RegisterMaskPair > *LiveUses=nullptr)
 Recede across the previous instruction. More...
 
void recedeSkipDebugValues ()
 Recede until we find an instruction which is not a DebugValue. More...
 
void advance ()
 Advance across the current instruction. More...
 
void advance (const RegisterOperands &RegOpers)
 Advance across the current instruction. More...
 
void closeRegion ()
 Finalize the region boundaries and recored live ins and live outs. More...
 
void initLiveThru (const RegPressureTracker &RPTracker)
 Initialize the LiveThru pressure set based on the untied defs found in RPTracker. More...
 
void initLiveThru (ArrayRef< unsigned > PressureSet)
 Copy an existing live thru pressure result. More...
 
ArrayRef< unsignedgetLiveThru () const
 
RegisterPressuregetPressure ()
 Get the resulting register pressure over the traversed region. More...
 
const RegisterPressuregetPressure () const
 
const std::vector< unsigned > & getRegSetPressureAtPos () const
 Get the register set pressure at the current position, which may be less than the pressure across the traversed region. More...
 
bool isTopClosed () const
 Does this pressure result have a valid top position and live ins. More...
 
bool isBottomClosed () const
 Does this pressure result have a valid bottom position and live outs. More...
 
void closeTop ()
 Set the boundary for the top of the region and summarize live ins. More...
 
void closeBottom ()
 Set the boundary for the bottom of the region and summarize live outs. More...
 
void getMaxUpwardPressureDelta (const MachineInstr *MI, PressureDiff *PDiff, RegPressureDelta &Delta, ArrayRef< PressureChange > CriticalPSets, ArrayRef< unsigned > MaxPressureLimit)
 Consider the pressure increase caused by traversing this instruction bottom-up. More...
 
void getUpwardPressureDelta (const MachineInstr *MI, PressureDiff &PDiff, RegPressureDelta &Delta, ArrayRef< PressureChange > CriticalPSets, ArrayRef< unsigned > MaxPressureLimit) const
 This is the fast version of querying register pressure that does not directly depend on current liveness. More...
 
void getMaxDownwardPressureDelta (const MachineInstr *MI, RegPressureDelta &Delta, ArrayRef< PressureChange > CriticalPSets, ArrayRef< unsigned > MaxPressureLimit)
 Consider the pressure increase caused by traversing this instruction top-down. More...
 
void getMaxPressureDelta (const MachineInstr *MI, RegPressureDelta &Delta, ArrayRef< PressureChange > CriticalPSets, ArrayRef< unsigned > MaxPressureLimit)
 Find the pressure set with the most change beyond its pressure limit after traversing this instruction either upward or downward depending on the closed end of the current region. More...
 
void getUpwardPressure (const MachineInstr *MI, std::vector< unsigned > &PressureResult, std::vector< unsigned > &MaxPressureResult)
 Get the pressure of each PSet after traversing this instruction bottom-up. More...
 
void getDownwardPressure (const MachineInstr *MI, std::vector< unsigned > &PressureResult, std::vector< unsigned > &MaxPressureResult)
 Get the pressure of each PSet after traversing this instruction top-down. More...
 
void getPressureAfterInst (const MachineInstr *MI, std::vector< unsigned > &PressureResult, std::vector< unsigned > &MaxPressureResult)
 
bool hasUntiedDef (unsigned VirtReg) const
 
void dump () const
 

Protected Member Functions

void discoverLiveOut (RegisterMaskPair Pair)
 Add Reg to the live out set and increase max pressure. More...
 
void discoverLiveIn (RegisterMaskPair Pair)
 Add Reg to the live in set and increase max pressure. More...
 
SlotIndex getCurrSlot () const
 Get the SlotIndex for the first nondebug instruction including or after the current position. More...
 
void increaseRegPressure (unsigned RegUnit, LaneBitmask PreviousMask, LaneBitmask NewMask)
 
void decreaseRegPressure (unsigned RegUnit, LaneBitmask PreviousMask, LaneBitmask NewMask)
 
void bumpDeadDefs (ArrayRef< RegisterMaskPair > DeadDefs)
 
void bumpUpwardPressure (const MachineInstr *MI)
 Record the upward impact of a single instruction on current register pressure. More...
 
void bumpDownwardPressure (const MachineInstr *MI)
 Record the downward impact of a single instruction on current register pressure. More...
 
void discoverLiveInOrOut (RegisterMaskPair Pair, SmallVectorImpl< RegisterMaskPair > &LiveInOrOut)
 
LaneBitmask getLastUsedLanes (unsigned RegUnit, SlotIndex Pos) const
 
LaneBitmask getLiveLanesAt (unsigned RegUnit, SlotIndex Pos) const
 
LaneBitmask getLiveThroughAt (unsigned RegUnit, SlotIndex Pos) const
 

Detailed Description

Track the current register pressure at some position in the instruction stream, and remember the high water mark within the region traversed.

This does not automatically consider live-through ranges. The client may independently adjust for global liveness.

Each RegPressureTracker only works within a MachineBasicBlock. Pressure can be tracked across a larger region by storing a RegisterPressure result at each block boundary and explicitly adjusting pressure to account for block live-in and live-out register sets.

RegPressureTracker holds a reference to a RegisterPressure result that it computes incrementally. During downward tracking, P.BottomIdx or P.BottomPos is invalid until it reaches the end of the block or closeRegion() is explicitly called. Similarly, P.TopIdx is invalid during upward tracking. Changing direction has the side effect of closing region, and traversing past TopIdx or BottomIdx reopens it.

Definition at line 360 of file RegisterPressure.h.

Constructor & Destructor Documentation

◆ RegPressureTracker() [1/2]

llvm::RegPressureTracker::RegPressureTracker ( IntervalPressure rp)
inline

Definition at line 400 of file RegisterPressure.h.

◆ RegPressureTracker() [2/2]

llvm::RegPressureTracker::RegPressureTracker ( RegionPressure rp)
inline

Definition at line 401 of file RegisterPressure.h.

References llvm::cl::init().

Member Function Documentation

◆ addLiveRegs()

void RegPressureTracker::addLiveRegs ( ArrayRef< RegisterMaskPair Regs)

Force liveness of virtual registers or physical register units.

Force liveness of registers.

Particularly useful to initialize the livein/out state of the tracker before the first call to advance/recede.

Definition at line 690 of file RegisterPressure.cpp.

References P.

Referenced by computeLiveOuts(), and isDefBetween().

◆ advance() [1/2]

void RegPressureTracker::advance ( )

Advance across the current instruction.

Definition at line 925 of file RegisterPressure.cpp.

References llvm::RegisterOperands::adjustLaneLiveness(), llvm::RegisterOperands::collect(), MI, and TRI.

◆ advance() [2/2]

void RegPressureTracker::advance ( const RegisterOperands RegOpers)

◆ bumpDeadDefs()

void RegPressureTracker::bumpDeadDefs ( ArrayRef< RegisterMaskPair DeadDefs)
protected

Definition at line 728 of file RegisterPressure.cpp.

References P, and Reg.

◆ bumpDownwardPressure()

void RegPressureTracker::bumpDownwardPressure ( const MachineInstr MI)
protected

Record the downward impact of a single instruction on current register pressure.

Unlike the advance/recede pressure tracking interface, this does not discover live in/outs.

This is intended for speculative queries. It leaves pressure inconsistent with the current position, so must be restored by the caller.

Definition at line 1261 of file RegisterPressure.cpp.

References llvm::RegisterOperands::adjustLaneLiveness(), assert(), llvm::RegisterOperands::collect(), llvm::RegisterOperands::DeadDefs, llvm::tgtok::Def, llvm::RegisterOperands::Defs, findUseBetween(), llvm::LiveIntervals::getInstructionIndex(), getMaxDownwardPressureDelta(), llvm::SlotIndex::getRegSlot(), llvm::MachineInstr::isDebugInstr(), llvm::RegisterMaskPair::LaneMask, llvm::LaneBitmask::none(), Reg, llvm::RegisterMaskPair::RegUnit, TRI, and llvm::RegisterOperands::Uses.

◆ bumpUpwardPressure()

void RegPressureTracker::bumpUpwardPressure ( const MachineInstr MI)
protected

Record the upward impact of a single instruction on current register pressure.

Unlike the advance/recede pressure tracking interface, this does not discover live in/outs.

This is intended for speculative queries. It leaves pressure inconsistent with the current position, so must be restored by the caller.

Definition at line 1020 of file RegisterPressure.cpp.

References llvm::RegisterOperands::adjustLaneLiveness(), assert(), llvm::RegisterOperands::collect(), llvm::RegisterOperands::DeadDefs, llvm::RegisterOperands::Defs, llvm::RegisterOperands::detectDeadDefs(), llvm::LiveIntervals::getInstructionIndex(), getMaxUpwardPressureDelta(), getRegLanes(), llvm::SlotIndex::getRegSlot(), llvm::MachineInstr::isDebugInstr(), llvm::RegisterMaskPair::LaneMask, P, Reg, llvm::RegisterMaskPair::RegUnit, TRI, and llvm::RegisterOperands::Uses.

◆ closeBottom()

void RegPressureTracker::closeBottom ( )

Set the boundary for the bottom of the region and summarize live outs.

Definition at line 316 of file RegisterPressure.cpp.

References assert(), and P.

Referenced by computeLiveOuts().

◆ closeRegion()

void RegPressureTracker::closeRegion ( )

Finalize the region boundaries and recored live ins and live outs.

Finalize the region boundaries and record live ins and live outs.

Definition at line 328 of file RegisterPressure.cpp.

References assert().

◆ closeTop()

void RegPressureTracker::closeTop ( )

Set the boundary for the top of the region and summarize live ins.

Definition at line 304 of file RegisterPressure.cpp.

References assert(), and P.

◆ decreaseRegPressure()

void RegPressureTracker::decreaseRegPressure ( unsigned  RegUnit,
LaneBitmask  PreviousMask,
LaneBitmask  NewMask 
)
protected

Definition at line 155 of file RegisterPressure.cpp.

References decreaseSetPressure(), and MRI.

◆ discoverLiveIn()

void RegPressureTracker::discoverLiveIn ( RegisterMaskPair  Pair)
protected

Add Reg to the live in set and increase max pressure.

Definition at line 720 of file RegisterPressure.cpp.

References P.

◆ discoverLiveInOrOut()

void RegPressureTracker::discoverLiveInOrOut ( RegisterMaskPair  Pair,
SmallVectorImpl< RegisterMaskPair > &  LiveInOrOut 
)
protected

◆ discoverLiveOut()

void RegPressureTracker::discoverLiveOut ( RegisterMaskPair  Pair)
protected

Add Reg to the live out set and increase max pressure.

Definition at line 724 of file RegisterPressure.cpp.

References P.

◆ dump()

LLVM_DUMP_METHOD void RegPressureTracker::dump ( ) const

Definition at line 118 of file RegisterPressure.cpp.

References llvm::dbgs(), llvm::dumpRegSetPressure(), LLVM_DUMP_METHOD, P, and TRI.

◆ getCurrSlot()

SlotIndex RegPressureTracker::getCurrSlot ( ) const
protected

Get the SlotIndex for the first nondebug instruction including or after the current position.

Definition at line 295 of file RegisterPressure.cpp.

References llvm::skipDebugInstructionsForward().

◆ getDownwardPressure()

void RegPressureTracker::getDownwardPressure ( const MachineInstr MI,
std::vector< unsigned > &  PressureResult,
std::vector< unsigned > &  MaxPressureResult 
)

Get the pressure of each PSet after traversing this instruction top-down.

Definition at line 1360 of file RegisterPressure.cpp.

References P.

Referenced by getUpwardPressure(), and llvm::GCNMaxOccupancySchedStrategy::initialize().

◆ getLastUsedLanes()

LaneBitmask RegPressureTracker::getLastUsedLanes ( unsigned  RegUnit,
SlotIndex  Pos 
) const
protected

◆ getLiveLanesAt()

LaneBitmask RegPressureTracker::getLiveLanesAt ( unsigned  RegUnit,
SlotIndex  Pos 
) const
protected

◆ getLiveThroughAt()

LaneBitmask RegPressureTracker::getLiveThroughAt ( unsigned  RegUnit,
SlotIndex  Pos 
) const
protected

◆ getLiveThru()

ArrayRef<unsigned> llvm::RegPressureTracker::getLiveThru ( ) const
inline

Definition at line 457 of file RegisterPressure.h.

◆ getMaxDownwardPressureDelta()

void RegPressureTracker::getMaxDownwardPressureDelta ( const MachineInstr MI,
RegPressureDelta Delta,
ArrayRef< PressureChange CriticalPSets,
ArrayRef< unsigned MaxPressureLimit 
)

Consider the pressure increase caused by traversing this instruction top-down.

Find the pressure set with the most change beyond its pressure limit based on the tracker's current pressure, and record the number of excess register units of that pressure set introduced by this instruction.

Find the register class with the most change in its pressure limit based on the tracker's current pressure, and return the number of excess register units of that pressure set introduced by this instruction.

This assumes that the current LiveIn set is sufficient.

This is expensive for an on-the-fly query because it calls bumpDownwardPressure to recompute the pressure sets based on current liveness. We don't yet have a fast version of downward pressure tracking analogous to getUpwardPressureDelta.

Definition at line 1321 of file RegisterPressure.cpp.

References assert(), computeExcessPressureDelta(), computeMaxPressureDelta(), llvm::RegPressureDelta::CriticalMax, llvm::RegPressureDelta::CurrentMax, llvm::PressureChange::getUnitInc(), getUpwardPressure(), and P.

Referenced by bumpDownwardPressure(), and llvm::GenericScheduler::initCandidate().

◆ getMaxPressureDelta()

void llvm::RegPressureTracker::getMaxPressureDelta ( const MachineInstr MI,
RegPressureDelta Delta,
ArrayRef< PressureChange CriticalPSets,
ArrayRef< unsigned MaxPressureLimit 
)
inline

Find the pressure set with the most change beyond its pressure limit after traversing this instruction either upward or downward depending on the closed end of the current region.

Definition at line 504 of file RegisterPressure.h.

References assert().

Referenced by llvm::ConvergingVLIWScheduler::readyQueueVerboseDump().

◆ getMaxUpwardPressureDelta()

void RegPressureTracker::getMaxUpwardPressureDelta ( const MachineInstr MI,
PressureDiff PDiff,
RegPressureDelta Delta,
ArrayRef< PressureChange CriticalPSets,
ArrayRef< unsigned MaxPressureLimit 
)

Consider the pressure increase caused by traversing this instruction bottom-up.

Find the pressure set with the most change beyond its pressure limit based on the tracker's current pressure, and record the number of excess register units of that pressure set introduced by this instruction.

Find the pressure set with the most change beyond its pressure limit based on the tracker's current pressure, and return the change in number of register units of that pressure set introduced by this instruction.

This assumes that the current LiveOut set is sufficient.

This is expensive for an on-the-fly query because it calls bumpUpwardPressure to recompute the pressure sets based on current liveness. This mainly exists to verify correctness, e.g. with -verify-misched. getUpwardPressureDelta is the fast version of this query that uses the per-SUnit cache of the PressureDiff.

Definition at line 1072 of file RegisterPressure.cpp.

References assert(), computeExcessPressureDelta(), computeMaxPressureDelta(), llvm::RegPressureDelta::CriticalMax, llvm::RegPressureDelta::CurrentMax, llvm::dbgs(), llvm::PressureDiff::dump(), llvm::RegPressureDelta::Excess, llvm::PressureChange::getPSet(), llvm::PressureChange::getUnitInc(), getUpwardPressureDelta(), llvm::PressureChange::isValid(), llvm_unreachable, MI, P, and TRI.

Referenced by bumpUpwardPressure(), computeLiveOuts(), and llvm::GenericScheduler::initCandidate().

◆ getPos()

MachineBasicBlock::const_iterator llvm::RegPressureTracker::getPos ( ) const
inline

Get the MI position corresponding to this register pressure.

Definition at line 416 of file RegisterPressure.h.

Referenced by llvm::ScheduleDAGInstrs::buildSchedGraph(), and llvm::SIScheduleDAGMI::schedule().

◆ getPressure() [1/2]

RegisterPressure& llvm::RegPressureTracker::getPressure ( )
inline

Get the resulting register pressure over the traversed region.

This result is complete if closeRegion() was explicitly invoked.

Definition at line 461 of file RegisterPressure.h.

References P.

◆ getPressure() [2/2]

const RegisterPressure& llvm::RegPressureTracker::getPressure ( ) const
inline

Definition at line 462 of file RegisterPressure.h.

References P.

◆ getPressureAfterInst()

void llvm::RegPressureTracker::getPressureAfterInst ( const MachineInstr MI,
std::vector< unsigned > &  PressureResult,
std::vector< unsigned > &  MaxPressureResult 
)
inline

Definition at line 527 of file RegisterPressure.h.

References assert().

◆ getRegSetPressureAtPos()

const std::vector<unsigned>& llvm::RegPressureTracker::getRegSetPressureAtPos ( ) const
inline

Get the register set pressure at the current position, which may be less than the pressure across the traversed region.

Definition at line 466 of file RegisterPressure.h.

References MI.

Referenced by llvm::GCNMaxOccupancySchedStrategy::initialize().

◆ getUpwardPressure()

void RegPressureTracker::getUpwardPressure ( const MachineInstr MI,
std::vector< unsigned > &  PressureResult,
std::vector< unsigned > &  MaxPressureResult 
)

Get the pressure of each PSet after traversing this instruction bottom-up.

Definition at line 1344 of file RegisterPressure.cpp.

References getDownwardPressure(), and P.

Referenced by getMaxDownwardPressureDelta(), and llvm::GCNMaxOccupancySchedStrategy::initialize().

◆ getUpwardPressureDelta()

void RegPressureTracker::getUpwardPressureDelta ( const MachineInstr MI,
PressureDiff PDiff,
RegPressureDelta Delta,
ArrayRef< PressureChange CriticalPSets,
ArrayRef< unsigned MaxPressureLimit 
) const

This is the fast version of querying register pressure that does not directly depend on current liveness.

Parameters
Deltacaptures information needed for heuristics.
CriticalPSetsAre the pressure sets that are known to exceed some limit within the region, not necessarily at the current position.
MaxPressureLimitIs the max pressure within the region, not necessarily at the current position.

Definition at line 1140 of file RegisterPressure.cpp.

References assert(), llvm::PressureDiff::begin(), llvm::RegPressureDelta::CriticalMax, llvm::RegPressureDelta::CurrentMax, llvm::PressureDiff::end(), llvm::RegPressureDelta::Excess, llvm::PressureChange::isValid(), llvm::max(), P, llvm::PressureChange::setUnitInc(), and llvm::ArrayRef< T >::size().

Referenced by getMaxUpwardPressureDelta(), and llvm::GenericScheduler::initCandidate().

◆ hasUntiedDef()

bool llvm::RegPressureTracker::hasUntiedDef ( unsigned  VirtReg) const
inline

◆ increaseRegPressure()

void RegPressureTracker::increaseRegPressure ( unsigned  RegUnit,
LaneBitmask  PreviousMask,
LaneBitmask  NewMask 
)
protected

◆ init()

void RegPressureTracker::init ( const MachineFunction mf,
const RegisterClassInfo rci,
const LiveIntervals lis,
const MachineBasicBlock mbb,
MachineBasicBlock::const_iterator  pos,
bool  TrackLaneMasks,
bool  TrackUntiedDefs 
)

Setup the RegPressureTracker.

TODO: Add support for pressure without LiveIntervals.

Definition at line 248 of file RegisterPressure.cpp.

References assert(), MRI, P, and TRI.

Referenced by computeLiveOuts(), and llvm::SIScheduleDAGMI::initRPTracker().

◆ initLiveThru() [1/2]

void RegPressureTracker::initLiveThru ( const RegPressureTracker RPTracker)

Initialize the LiveThru pressure set based on the untied defs found in RPTracker.

The register tracker is unaware of global liveness so ignores normal live-thru ranges.

However, two-address or coalesced chains can also lead to live ranges with no holes. Count these to inform heuristics that we can never drop below this pressure.

Definition at line 344 of file RegisterPressure.cpp.

References assert(), llvm::LaneBitmask::getNone(), hasUntiedDef(), increaseSetPressure(), llvm::TargetRegisterInfo::isVirtualRegister(), llvm::RegisterMaskPair::LaneMask, MRI, P, llvm::RegisterMaskPair::RegUnit, and TRI.

◆ initLiveThru() [2/2]

void llvm::RegPressureTracker::initLiveThru ( ArrayRef< unsigned PressureSet)
inline

Copy an existing live thru pressure result.

Definition at line 453 of file RegisterPressure.h.

References llvm::ArrayRef< T >::begin(), and llvm::ArrayRef< T >::end().

◆ isBottomClosed()

bool RegPressureTracker::isBottomClosed ( ) const

Does this pressure result have a valid bottom position and live outs.

Definition at line 288 of file RegisterPressure.cpp.

References P.

◆ isTopClosed()

bool RegPressureTracker::isTopClosed ( ) const

Does this pressure result have a valid top position and live ins.

Definition at line 280 of file RegisterPressure.cpp.

References P.

◆ recede() [1/2]

void RegPressureTracker::recede ( SmallVectorImpl< RegisterMaskPair > *  LiveUses = nullptr)

◆ recede() [2/2]

void RegPressureTracker::recede ( const RegisterOperands RegOpers,
SmallVectorImpl< RegisterMaskPair > *  LiveUses = nullptr 
)

Recede across the previous instruction.

This "low-level" variant assumes that recedeSkipDebugValues() was called previously and takes precomputed RegisterOperands for the instruction.

If LiveUses is provided, record any RegUnits that are made live by the current instruction's uses. This includes registers that are both defined and used by the instruction. If a pressure difference pointer is provided record the changes is pressure caused by this instruction independent of liveness.

Definition at line 748 of file RegisterPressure.cpp.

References addRegLanes(), llvm::LaneBitmask::any(), assert(), llvm::RegisterOperands::DeadDefs, llvm::tgtok::Def, llvm::RegisterOperands::Defs, llvm::SmallVectorTemplateCommon< T >::end(), llvm::find_if(), llvm::LiveIntervals::getInstructionIndex(), llvm::LaneBitmask::getNone(), llvm::SlotIndex::getRegSlot(), I, increaseSetPressure(), llvm::TargetRegisterInfo::isVirtualRegister(), llvm::RegisterMaskPair::LaneMask, llvm::LaneBitmask::none(), Other, Reg, llvm::RegisterMaskPair::RegUnit, removeRegLanes(), setRegZero(), and llvm::RegisterOperands::Uses.

◆ recedeSkipDebugValues()

void RegPressureTracker::recedeSkipDebugValues ( )

Recede until we find an instruction which is not a DebugValue.

Definition at line 836 of file RegisterPressure.cpp.

References assert(), llvm::LiveIntervals::getInstructionIndex(), llvm::SlotIndex::getRegSlot(), P, and llvm::skipDebugInstructionsBackward().

Referenced by llvm::ScheduleDAGInstrs::buildSchedGraph().

◆ reset()

void RegPressureTracker::reset ( )

Definition at line 228 of file RegisterPressure.cpp.

References P.

◆ setPos()

void llvm::RegPressureTracker::setPos ( MachineBasicBlock::const_iterator  Pos)
inline

Definition at line 422 of file RegisterPressure.h.

Referenced by computeLiveOuts(), and llvm::SIScheduleDAGMI::schedule().


The documentation for this class was generated from the following files: