Inheritance diagram for llvm::InnerLoopUnroller:
Collaboration diagram for llvm::InnerLoopUnroller:
Public Member Functions | |
| InnerLoopUnroller (Loop *OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo *LI, DominatorTree *DT, const TargetLibraryInfo *TLI, const TargetTransformInfo *TTI, AssumptionCache *AC, OptimizationRemarkEmitter *ORE, unsigned UnrollFactor, LoopVectorizationLegality *LVL, LoopVectorizationCostModel *CM) | |
 Public Member Functions inherited from llvm::InnerLoopVectorizer | |
| InnerLoopVectorizer (Loop *OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo *LI, DominatorTree *DT, const TargetLibraryInfo *TLI, const TargetTransformInfo *TTI, AssumptionCache *AC, OptimizationRemarkEmitter *ORE, unsigned VecWidth, unsigned UnrollFactor, LoopVectorizationLegality *LVL, LoopVectorizationCostModel *CM) | |
| virtual | ~InnerLoopVectorizer ()=default |
| BasicBlock * | createVectorizedLoopSkeleton () |
| Create a new empty loop. More... | |
| void | widenInstruction (Instruction &I) |
| Widen a single instruction within the innermost loop. More... | |
| void | fixVectorizedLoop () |
| Fix the vectorized code, taking care of header phi's, live-outs, and more. More... | |
| bool | areSafetyChecksAdded () |
| void | widenPHIInstruction (Instruction *PN, unsigned UF, unsigned VF) |
| Vectorize a single PHINode in a block. More... | |
| void | scalarizeInstruction (Instruction *Instr, const VPIteration &Instance, bool IfPredicateInstr) |
| A helper function to scalarize a single Instruction in the innermost loop. More... | |
| void | widenIntOrFpInduction (PHINode *IV, TruncInst *Trunc=nullptr) |
Widen an integer or floating-point induction variable IV. More... | |
| Value * | getOrCreateVectorValue (Value *V, unsigned Part) |
| getOrCreateVectorValue and getOrCreateScalarValue coordinate to generate a vector or scalar value on-demand if one is not yet available. More... | |
| Value * | getOrCreateScalarValue (Value *V, const VPIteration &Instance) |
Return a value in the new loop corresponding to V from the original loop at unroll and vector indices Instance. More... | |
| void | packScalarIntoVectorValue (Value *V, const VPIteration &Instance) |
Construct the vector value of a scalarized value V one lane at a time. More... | |
| void | vectorizeInterleaveGroup (Instruction *Instr, VectorParts *BlockInMask=nullptr) |
Try to vectorize the interleaved access group that Instr belongs to, optionally masking the vector operations if BlockInMask is non-null. More... | |
| void | vectorizeMemoryInstruction (Instruction *Instr, VectorParts *BlockInMask=nullptr) |
Vectorize Load and Store instructions, optionally masking the vector operations if BlockInMask is non-null. More... | |
| void | setDebugLocFromInst (IRBuilder<> &B, const Value *Ptr) |
| Set the debug location in the builder using the debug location in the instruction. More... | |
| void | fixNonInductionPHIs (void) |
| Fix the non-induction PHIs in the OrigPHIsToFix vector. More... | |
Additional Inherited Members | |
| Public Types inherited from llvm::InnerLoopVectorizer | |
| using | VectorParts = SmallVector< Value *, 2 > |
| A type for vectorized values in the new loop. More... | |
| Protected Types inherited from llvm::InnerLoopVectorizer | |
| using | PhiVector = SmallVector< PHINode *, 4 > |
| A small list of PHINodes. More... | |
| using | ScalarParts = SmallVector< SmallVector< Value *, 4 >, 2 > |
| A type for scalarized values in the new loop. More... | |
| Protected Member Functions inherited from llvm::InnerLoopVectorizer | |
| void | fixupIVUsers (PHINode *OrigPhi, const InductionDescriptor &II, Value *CountRoundDown, Value *EndValue, BasicBlock *MiddleBlock) |
| Set up the values of the IVs correctly when exiting the vector loop. More... | |
| PHINode * | createInductionVariable (Loop *L, Value *Start, Value *End, Value *Step, Instruction *DL) |
| Create a new induction variable inside L. More... | |
| void | fixCrossIterationPHIs () |
| Handle all cross-iteration phis in the header. More... | |
| void | fixFirstOrderRecurrence (PHINode *Phi) |
| Fix a first-order recurrence. More... | |
| void | fixReduction (PHINode *Phi) |
| Fix a reduction cross-iteration phi. More... | |
| void | fixLCSSAPHIs () |
| The Loop exit block may have single value PHI nodes with some incoming value. More... | |
| void | sinkScalarOperands (Instruction *PredInst) |
| Iteratively sink the scalarized operands of a predicated instruction into the block that was created for it. More... | |
| void | truncateToMinimalBitwidths () |
| Shrinks vector element sizes to the smallest bitwidth they can be legally represented as. More... | |
| void | updateAnalysis () |
| Insert the new loop to the loop hierarchy and pass manager and update the analysis passes. More... | |
| void | buildScalarSteps (Value *ScalarIV, Value *Step, Instruction *EntryVal, const InductionDescriptor &ID) |
| Compute scalar induction steps. More... | |
| void | createVectorIntOrFpInductionPHI (const InductionDescriptor &II, Value *Step, Instruction *EntryVal) |
| Create a vector induction phi node based on an existing scalar one. More... | |
| bool | shouldScalarizeInstruction (Instruction *I) const |
Returns true if an instruction I should be scalarized instead of vectorized for the chosen vectorization factor. More... | |
| bool | needsScalarInduction (Instruction *IV) const |
Returns true if we should generate a scalar version of IV. More... | |
| void | recordVectorLoopValueForInductionCast (const InductionDescriptor &ID, const Instruction *EntryVal, Value *VectorLoopValue, unsigned Part, unsigned Lane=UINT_MAX) |
If there is a cast involved in the induction variable ID, which should be ignored in the vectorized loop body, this function records the VectorLoopValue of the respective Phi also as the VectorLoopValue of the cast. More... | |
| Value * | getOrCreateTripCount (Loop *NewLoop) |
| Returns (and creates if needed) the original loop trip count. More... | |
| Value * | getOrCreateVectorTripCount (Loop *NewLoop) |
| Returns (and creates if needed) the trip count of the widened loop. More... | |
| Value * | createBitOrPointerCast (Value *V, VectorType *DstVTy, const DataLayout &DL) |
| Returns a bitcasted value to the requested vector type. More... | |
| void | emitMinimumIterationCountCheck (Loop *L, BasicBlock *Bypass) |
| Emit a bypass check to see if the vector trip count is zero, including if it overflows. More... | |
| void | emitSCEVChecks (Loop *L, BasicBlock *Bypass) |
| Emit a bypass check to see if all of the SCEV assumptions we've had to make are correct. More... | |
| void | emitMemRuntimeChecks (Loop *L, BasicBlock *Bypass) |
| Emit bypass checks to check any memory assumptions we may have made. More... | |
| Value * | emitTransformedIndex (IRBuilder<> &B, Value *Index, ScalarEvolution *SE, const DataLayout &DL, const InductionDescriptor &ID) const |
| Compute the transformed value of Index at offset StartValue using step StepValue. More... | |
| void | addNewMetadata (Instruction *To, const Instruction *Orig) |
Add additional metadata to To that was not present on Orig. More... | |
| void | addMetadata (Instruction *To, Instruction *From) |
| Add metadata from one instruction to another. More... | |
| void | addMetadata (ArrayRef< Value *> To, Instruction *From) |
| Similar to the previous function but it adds the metadata to a vector of instructions. More... | |
| Protected Attributes inherited from llvm::InnerLoopVectorizer | |
| Loop * | OrigLoop |
| The original loop. More... | |
| PredicatedScalarEvolution & | PSE |
| A wrapper around ScalarEvolution used to add runtime SCEV checks. More... | |
| LoopInfo * | LI |
| Loop Info. More... | |
| DominatorTree * | DT |
| Dominator Tree. More... | |
| AliasAnalysis * | AA |
| Alias Analysis. More... | |
| const TargetLibraryInfo * | TLI |
| Target Library Info. More... | |
| const TargetTransformInfo * | TTI |
| Target Transform Info. More... | |
| AssumptionCache * | AC |
| Assumption Cache. More... | |
| OptimizationRemarkEmitter * | ORE |
| Interface to emit optimization remarks. More... | |
| std::unique_ptr< LoopVersioning > | LVer |
| LoopVersioning. More... | |
| unsigned | VF |
| The vectorization SIMD factor to use. More... | |
| unsigned | UF |
| The vectorization unroll factor to use. More... | |
| IRBuilder | Builder |
| The builder that we use. More... | |
| BasicBlock * | LoopVectorPreHeader |
| The vector-loop preheader. More... | |
| BasicBlock * | LoopScalarPreHeader |
| The scalar-loop preheader. More... | |
| BasicBlock * | LoopMiddleBlock |
| Middle Block between the vector and the scalar. More... | |
| BasicBlock * | LoopExitBlock |
| The ExitBlock of the scalar loop. More... | |
| BasicBlock * | LoopVectorBody |
| The vector loop body. More... | |
| BasicBlock * | LoopScalarBody |
| The scalar loop body. More... | |
| SmallVector< BasicBlock *, 4 > | LoopBypassBlocks |
| A list of all bypass blocks. The first block is the entry of the loop. More... | |
| PHINode * | Induction = nullptr |
| The new Induction variable which was added to the new block. More... | |
| PHINode * | OldInduction = nullptr |
| The induction variable of the old basic block. More... | |
| VectorizerValueMap | VectorLoopValueMap |
| Maps values from the original loop to their corresponding values in the vectorized loop. More... | |
| SmallVector< Instruction *, 4 > | PredicatedInstructions |
| Store instructions that were predicated. More... | |
| Value * | TripCount = nullptr |
| Trip count of the original loop. More... | |
| Value * | VectorTripCount = nullptr |
| Trip count of the widened loop (TripCount - TripCount % (VF*UF)) More... | |
| LoopVectorizationLegality * | Legal |
| The legality analysis. More... | |
| LoopVectorizationCostModel * | Cost |
| The profitablity analysis. More... | |
| bool | AddedSafetyChecks = false |
| DenseMap< PHINode *, Value * > | IVEndValues |
| SmallVector< PHINode *, 8 > | OrigPHIsToFix |
Detailed Description
Definition at line 717 of file LoopVectorize.cpp.
Constructor & Destructor Documentation
◆ InnerLoopUnroller()
|
inline |
Definition at line 719 of file LoopVectorize.cpp.
References llvm::InnerLoopVectorizer::getBroadcastInstrs(), llvm::InnerLoopVectorizer::getStepVector(), and llvm::InnerLoopVectorizer::reverseVector().
The documentation for this class was generated from the following file:
- lib/Transforms/Vectorize/LoopVectorize.cpp
